Jun 02 04:42:32.858084 ip-10-0-135-23 systemd[1]: Starting Kubernetes Kubelet...
Jun 02 04:42:33.222425 ip-10-0-135-23 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 02 04:42:33.222425 ip-10-0-135-23 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jun 02 04:42:33.222425 ip-10-0-135-23 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 02 04:42:33.222425 ip-10-0-135-23 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jun 02 04:42:33.222425 ip-10-0-135-23 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 02 04:42:33.223681 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.223592    2577 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jun 02 04:42:33.227426 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227403    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 04:42:33.227426 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227421    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 04:42:33.227426 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227427    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 04:42:33.227426 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227431    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227436    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227440    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227444    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227448    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227452    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227456    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227460    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227464    2577 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227468    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227472    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227476    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227480    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227485    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227489    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227492    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227502    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227506    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227510    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227514    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 04:42:33.227667 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227519    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227522    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227526    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227530    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227534    2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227538    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227542    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227546    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227550    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227554    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227559    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227566    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227573    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227578    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227584    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227589    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227593    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227598    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227603    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 04:42:33.228545 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227608    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227612    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227616    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227623    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227627    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227632    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227636    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227641    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227645    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227649    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227653    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227657    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227662    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227666    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227670    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227674    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227678    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227682    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227687    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227691    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 04:42:33.229292 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227697    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227701    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227706    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227710    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227714    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227718    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227722    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227728    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227732    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227736    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227741    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227745    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227749    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227753    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227758    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227762    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227767    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227772    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227776    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227782    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 04:42:33.229795 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227791    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227796    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227800    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.227804    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228448    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228458    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228462    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228466    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228470    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228474    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228479    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228483    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228487    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228491    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228497    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228503    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228507    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228511    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228515    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228521    2577 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 04:42:33.230537 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228525    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228530    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228534    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228538    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228542    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228546    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228550    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228554    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228560    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228564    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228568    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228572    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228576    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228580    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228584    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228588    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228593    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228597    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228612    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228617    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 04:42:33.231395 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228621    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228625    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228632    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228638    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228643    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228648    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228652    2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228656    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228661    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228665    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228669    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228674    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228679    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228683    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228687    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228691    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228696    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228700    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228704    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 04:42:33.232059 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228708    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228712    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228717    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228721    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228725    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228729    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228733    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228737    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228741    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228746    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228750    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228753    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228757    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228761    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228765    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228769    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228774    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228778    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228783    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228787    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 04:42:33.232701 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228793    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228797    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228801    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228805    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228826    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228831    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228835    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228839    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228844    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228848    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.228852    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230000    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230016    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230028    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230034    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230042    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230047    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230054    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230061    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230066    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jun 02 04:42:33.233206 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230072    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230077    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230082    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230088    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230093    2577 flags.go:64] FLAG: --cgroup-root=""
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230097    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230103    2577 flags.go:64] FLAG: --client-ca-file=""
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230107    2577 flags.go:64] FLAG: --cloud-config=""
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230112    2577 flags.go:64] FLAG: --cloud-provider="external"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230117    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230123    2577 flags.go:64] FLAG: --cluster-domain=""
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230129    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230134    2577 flags.go:64] FLAG: --config-dir=""
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230139    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230145    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230151    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230156    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230161    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230167    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230172    2577 flags.go:64] FLAG: --contention-profiling="false"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230177    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230182    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230187    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230192    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230199    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jun 02 04:42:33.233785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230204    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230209    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230214    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230220    2577 flags.go:64] FLAG: --enable-server="true"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230224    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230232    2577 flags.go:64] FLAG: --event-burst="100"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230237    2577 flags.go:64] FLAG: --event-qps="50"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230242    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230259    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230264    2577 flags.go:64] FLAG: --eviction-hard=""
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230270    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230275    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230280    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230284    2577 flags.go:64] FLAG: --eviction-soft=""
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230289    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230294    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230299    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230307    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230312    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230317    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230321    2577 flags.go:64] FLAG: --feature-gates=""
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230328    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230333    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230338    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230343    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230348    2577 flags.go:64] FLAG: --healthz-port="10248"
Jun 02 04:42:33.234416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230353    2577 flags.go:64] FLAG: --help="false"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230358    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230363    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230368    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230373    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230379    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230384    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230389    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230394    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230398    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230404    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230409    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230415    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230419    2577 flags.go:64] FLAG: --kube-reserved=""
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230424    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230429    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230434    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230439    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230443    2577 flags.go:64] FLAG: --lock-file=""
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230448    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230452    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230458    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230467    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jun 02 04:42:33.235053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230474    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230480    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230484    2577 flags.go:64] FLAG: --logging-format="text"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230489    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230494    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230499    2577 flags.go:64] FLAG: --manifest-url=""
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230504    2577 flags.go:64] FLAG: --manifest-url-header=""
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230511    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230516    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230523    2577 flags.go:64] FLAG: --max-pods="110"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230528    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230532    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230537    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230542    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230546    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230551    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230556    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230567    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230572    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230577    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230583    2577 flags.go:64] FLAG: --pod-cidr=""
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230588    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230595    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230601    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jun 02 04:42:33.235592 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230605    2577 flags.go:64] FLAG: --pods-per-core="0"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230610    2577 flags.go:64] FLAG: --port="10250"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230616    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230620    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0dbfe1314eb1b0a97"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230626    2577 flags.go:64] FLAG: --qos-reserved=""
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230631    2577 flags.go:64] FLAG: --read-only-port="10255"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230636    2577 flags.go:64] FLAG: --register-node="true"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230640    2577 flags.go:64] FLAG: --register-schedulable="true"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230645    2577 flags.go:64] FLAG: --register-with-taints=""
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230652    2577 flags.go:64] FLAG: --registry-burst="10"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230657    2577 flags.go:64] FLAG: --registry-qps="5"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230662    2577 flags.go:64] FLAG: --reserved-cpus=""
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230666    2577 flags.go:64] FLAG: --reserved-memory=""
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230673    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230678    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230682    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230687    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230692    2577 flags.go:64] FLAG: --runonce="false"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230696    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230701    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230706    2577 flags.go:64] FLAG: --seccomp-default="false"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230711    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230716    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230721    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230726    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230731    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jun 02 04:42:33.236191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230736    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230741    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230746    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230751    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230756    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230761    2577 flags.go:64] FLAG: --system-cgroups=""
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230765    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230774    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230778    2577 flags.go:64] FLAG: --tls-cert-file=""
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230783    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230788    2577 flags.go:64] FLAG: --tls-min-version=""
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230793    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230798    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230802    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230807    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230829    2577 flags.go:64] FLAG: --v="2"
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230836    2577 flags.go:64] FLAG: --version="false"
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230843    2577 flags.go:64] FLAG: --vmodule=""
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230849    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.230855    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231025    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231034    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231040    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231045    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 04:42:33.236900 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231050    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231054    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231059    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231063    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231067    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231072    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231076    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231080    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231085    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231089    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231093    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231098    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231103    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231108    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231113    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231117    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231121    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231125    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231129    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231133    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 04:42:33.237463 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231138    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231142    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231146    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231151    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231155    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231159    2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231164    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231168    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231172    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231178    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231182    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231186    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231190    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231194    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231198    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231202    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231206    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231213    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231218    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231222    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 04:42:33.238015 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231227    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231231    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231234    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231238    2577 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231243    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231248    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231253    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231257    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231261    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231265    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231269    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231273    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231277    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231281    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231285    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231289    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231294    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231298    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231303    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 04:42:33.238505 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231307    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231311    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231320    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231325    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231329    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231333    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231338    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231343    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231347    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231351    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231355    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231359    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231363    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231368    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231372    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231376    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231380    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231384    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231391    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 04:42:33.238996 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231398    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 04:42:33.239459 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231402    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 04:42:33.239459 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231407    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 04:42:33.239459 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.231411    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 04:42:33.239459 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.231418    2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Jun 02 04:42:33.239459 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.237587    2577 server.go:530] "Kubelet version" kubeletVersion="v1.33.12"
Jun 02 04:42:33.239459 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.237603    2577 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jun 02 04:42:33.239459 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237647    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 04:42:33.239459 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237652    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 04:42:33.239459 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237655    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 04:42:33.239459 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237658    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 04:42:33.239459 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237661    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 04:42:33.239459 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237664    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 04:42:33.239459 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237667    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 04:42:33.239459 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237670    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 04:42:33.239459 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237673    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 04:42:33.239459 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237676    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237678    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237681    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237684    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237686    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237689    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237692    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237695    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237698    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237703    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237706    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237708    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237711    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237713    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237716    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237719    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237721    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237724    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237727    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237730    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 04:42:33.239878 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237732    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237735    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237739    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237741    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237744    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237746    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237749    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237751    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237754    2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237757    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237759    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237762    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237764    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237767    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237769    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237772    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237774    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237777    2577 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237779    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237782    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 04:42:33.240371 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237784    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237787    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237789    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237792    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237794    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237797    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237800    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237802    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237805    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237822    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237826    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237829    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237832    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237834    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237837    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237841    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237843    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237846    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237848    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237851    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 04:42:33.240876 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237854    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 04:42:33.241351 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237856    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 04:42:33.241351 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237859    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 04:42:33.241351 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237861    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 04:42:33.241351 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237864    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 04:42:33.241351 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237866    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 04:42:33.241351 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237869    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 04:42:33.241351 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237872    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 04:42:33.241351 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237874    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 04:42:33.241351 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237877    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 04:42:33.241351 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237879    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 04:42:33.241351 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237882    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 04:42:33.241351 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237886    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 04:42:33.241351 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237889    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 04:42:33.241351 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237892    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 04:42:33.241351 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237895    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 04:42:33.241351 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237898    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 04:42:33.241721 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.237903    2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Jun 02 04:42:33.241721 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.237995    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jun 02 04:42:33.241721 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238001    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jun 02 04:42:33.241721 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238004    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jun 02 04:42:33.241721 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238007    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jun 02 04:42:33.241721 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238010    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jun 02 04:42:33.241721 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238012    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jun 02 04:42:33.241721 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238015    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jun 02 04:42:33.241721 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238018    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jun 02 04:42:33.241721 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238020    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
Jun 02 04:42:33.241721 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238023    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jun 02 04:42:33.241721 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238026    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jun 02 04:42:33.241721 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238029    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jun 02 04:42:33.241721 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238031    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jun 02 04:42:33.241721 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238034    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238037    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238040    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238042    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238045    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238047    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238050    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238052    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238055    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238057    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238060    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238064    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238068    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238070    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238073    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238075    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238078    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238081    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238083    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jun 02 04:42:33.242138 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238086    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238088    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238092    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238096    2577 feature_gate.go:328] unrecognized feature gate: Example
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238099    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238101    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238104    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238107    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238110    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238112    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238115    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238118    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238120    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238123    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238125    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238128    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238130    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238133    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238135    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238137    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jun 02 04:42:33.242579 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238140    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238142    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238145    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238147    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238150    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238153    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238155    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238158    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238160    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238163    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238166    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238168    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238171    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238174    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238176    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238179    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238182    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238184    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238187    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238189    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jun 02 04:42:33.243070 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238191    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jun 02 04:42:33.243549 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238194    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jun 02 04:42:33.243549 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238196    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jun 02 04:42:33.243549 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238199    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jun 02 04:42:33.243549 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238202    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jun 02 04:42:33.243549 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238204    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jun 02 04:42:33.243549 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238206    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jun 02 04:42:33.243549 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238209    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jun 02 04:42:33.243549 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238211    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jun 02 04:42:33.243549 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238214    2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
Jun 02 04:42:33.243549 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238216    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jun 02 04:42:33.243549 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238219    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jun 02 04:42:33.243549 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238221    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jun 02 04:42:33.243549 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:33.238224    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jun 02 04:42:33.243549 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.238229    2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
Jun 02 04:42:33.243549 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.238897    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jun 02 04:42:33.243933 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.242584    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jun 02 04:42:33.243933 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.243381    2577 server.go:1019] "Starting client certificate rotation"
Jun 02 04:42:33.243933 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.243476    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jun 02 04:42:33.243933 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.243508    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jun 02 04:42:33.264925 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.264908    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jun 02 04:42:33.268141 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.268128    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jun 02 04:42:33.282312 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.282296    2577 log.go:25] "Validated CRI v1 runtime API"
Jun 02 04:42:33.287623 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.287608    2577 log.go:25] "Validated CRI v1 image API"
Jun 02 04:42:33.289506 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.289488    2577 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jun 02 04:42:33.291633 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.291615    2577 fs.go:135] Filesystem UUIDs: map[035fc7b9-3285-409d-84c8-0efdd9643b7d:/dev/nvme0n1p3 22e60815-dc90-4ce5-afd9-f2ddef83463b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jun 02 04:42:33.291687 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.291633    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jun 02 04:42:33.293860 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.293841    2577 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jun 02 04:42:33.296446 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.296341    2577 manager.go:217] Machine: {Timestamp:2026-06-02 04:42:33.295767869 +0000 UTC m=+0.336698831 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099372 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec287b83471aa8dedb3fea6088fd79f2 SystemUUID:ec287b83-471a-a8de-db3f-ea6088fd79f2 BootID:4d4df3f7-f847-4f76-b762-0657b2d06fb8 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:76:f7:8f:46:df Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:76:f7:8f:46:df Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:08:13:10:ba:ca Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jun 02 04:42:33.297002 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.296990    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jun 02 04:42:33.297072 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.297061    2577 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jun 02 04:42:33.300021 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.299992    2577 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jun 02 04:42:33.300151 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.300023    2577 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-23.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jun 02 04:42:33.300202 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.300161    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jun 02 04:42:33.300202 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.300170    2577 container_manager_linux.go:306] "Creating device plugin manager"
Jun 02 04:42:33.300202 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.300177    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jun 02 04:42:33.301507 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.301496    2577 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jun 02 04:42:33.302858 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.302848    2577 state_mem.go:36] "Initialized new in-memory state store"
Jun 02 04:42:33.302997 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.302988    2577 server.go:1267] "Using root directory" path="/var/lib/kubelet"
Jun 02 04:42:33.304987 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.304977    2577 kubelet.go:491] "Attempting to sync node with API server"
Jun 02 04:42:33.305023 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.304994    2577 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
Jun 02 04:42:33.305023 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.305006    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jun 02 04:42:33.305023 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.305015    2577 kubelet.go:397] "Adding apiserver pod source"
Jun 02 04:42:33.305023 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.305022    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jun 02 04:42:33.305974 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.305962    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jun 02 04:42:33.306021 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.305981    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jun 02 04:42:33.308587 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.308570    2577 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1"
Jun 02 04:42:33.310393 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.310379    2577 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jun 02 04:42:33.311714 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.311694    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-txhrr"
Jun 02 04:42:33.312527 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.312515    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jun 02 04:42:33.312577 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.312536    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jun 02 04:42:33.312577 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.312546    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jun 02 04:42:33.312577 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.312552    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jun 02 04:42:33.312577 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.312560    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jun 02 04:42:33.312577 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.312566    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jun 02 04:42:33.312577 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.312571    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jun 02 04:42:33.312577 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.312577    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jun 02 04:42:33.312774 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.312583    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jun 02 04:42:33.312774 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.312595    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jun 02 04:42:33.312774 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.312617    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jun 02 04:42:33.312774 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.312623    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jun 02 04:42:33.313392 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.313383    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jun 02 04:42:33.313430 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.313393    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jun 02 04:42:33.316582 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.316566    2577 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-23.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jun 02 04:42:33.316582 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.316573    2577 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jun 02 04:42:33.316694 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.316657    2577 watchdog_linux.go:99] "Systemd watchdog is not enabled"
Jun 02 04:42:33.316735 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.316703    2577 server.go:1295] "Started kubelet"
Jun 02 04:42:33.316785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.316759    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jun 02 04:42:33.316876 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.316788    2577 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jun 02 04:42:33.316933 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.316893    2577 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-23.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jun 02 04:42:33.316989 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.316970    2577 server_v1.go:47] "podresources" method="list" useActivePods=true
Jun 02 04:42:33.317485 ip-10-0-135-23 systemd[1]: Started Kubernetes Kubelet.
Jun 02 04:42:33.318194 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.318171    2577 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jun 02 04:42:33.320447 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.320348    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-txhrr"
Jun 02 04:42:33.320952 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.320935    2577 server.go:317] "Adding debug handlers to kubelet server"
Jun 02 04:42:33.325182 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.324216    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-23.ec2.internal.18b529f3754cf83b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-23.ec2.internal,UID:ip-10-0-135-23.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-23.ec2.internal,},FirstTimestamp:2026-06-02 04:42:33.316669499 +0000 UTC m=+0.357600462,LastTimestamp:2026-06-02 04:42:33.316669499 +0000 UTC m=+0.357600462,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-23.ec2.internal,}"
Jun 02 04:42:33.325294 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.325280    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jun 02 04:42:33.325294 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.325285    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jun 02 04:42:33.325496 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.325477    2577 volume_manager.go:295] "The desired_state_of_world populator starts"
Jun 02 04:42:33.325682 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.325664    2577 volume_manager.go:297] "Starting Kubelet Volume Manager"
Jun 02 04:42:33.325755 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.325713    2577 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
Jun 02 04:42:33.325853 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.325840    2577 reconstruct.go:97] "Volume reconstruction finished"
Jun 02 04:42:33.325853 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.325853    2577 reconciler.go:26] "Reconciler: start to sync state"
Jun 02 04:42:33.326144 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.326128    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-23.ec2.internal\" not found"
Jun 02 04:42:33.326633 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.326614    2577 factory.go:55] Registering systemd factory
Jun 02 04:42:33.326633 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.326634    2577 factory.go:223] Registration of the systemd container factory successfully
Jun 02 04:42:33.326878 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.326863    2577 factory.go:153] Registering CRI-O factory
Jun 02 04:42:33.326878 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.326871    2577 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jun 02 04:42:33.326984 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.326881    2577 factory.go:223] Registration of the crio container factory successfully
Jun 02 04:42:33.326984 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.326941    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jun 02 04:42:33.326984 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.326960    2577 factory.go:103] Registering Raw factory
Jun 02 04:42:33.326984 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.326973    2577 manager.go:1196] Started watching for new ooms in manager
Jun 02 04:42:33.327629 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.327613    2577 manager.go:319] Starting recovery of all containers
Jun 02 04:42:33.329079 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.329061    2577 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 04:42:33.336731 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.336712    2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-23.ec2.internal\" not found" node="ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.337873 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.337860    2577 manager.go:324] Recovery completed
Jun 02 04:42:33.339176 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.339159    2577 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jun 02 04:42:33.341998 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.341986    2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 04:42:33.343713 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.343698    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 04:42:33.343803 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.343728    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 04:42:33.343803 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.343744    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasSufficientPID"
Jun 02 04:42:33.344323 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.344309    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jun 02 04:42:33.344323 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.344323    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jun 02 04:42:33.344452 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.344351    2577 state_mem.go:36] "Initialized new in-memory state store"
Jun 02 04:42:33.346826 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.346719    2577 policy_none.go:49] "None policy: Start"
Jun 02 04:42:33.346907 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.346834    2577 memory_manager.go:186] "Starting memorymanager" policy="None"
Jun 02 04:42:33.346907 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.346850    2577 state_mem.go:35] "Initializing new in-memory state store"
Jun 02 04:42:33.390351 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.390339    2577 manager.go:341] "Starting Device Plugin manager"
Jun 02 04:42:33.390458 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.390366    2577 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jun 02 04:42:33.390458 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.390374    2577 server.go:85] "Starting device plugin registration server"
Jun 02 04:42:33.390580 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.390569    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jun 02 04:42:33.390627 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.390582    2577 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jun 02 04:42:33.390670 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.390660    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jun 02 04:42:33.390742 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.390728    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jun 02 04:42:33.390786 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.390742    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jun 02 04:42:33.391194 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.391176    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jun 02 04:42:33.391274 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.391214    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-23.ec2.internal\" not found"
Jun 02 04:42:33.438120 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.438096    2577 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
Jun 02 04:42:33.439179 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.439163    2577 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
Jun 02 04:42:33.439179 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.439179    2577 status_manager.go:230] "Starting to sync pod status with apiserver"
Jun 02 04:42:33.439344 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.439194    2577 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jun 02 04:42:33.439344 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.439204    2577 kubelet.go:2451] "Starting kubelet main sync loop"
Jun 02 04:42:33.439344 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.439247    2577 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jun 02 04:42:33.443062 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.443042    2577 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 04:42:33.490979 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.490963    2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 04:42:33.491710 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.491696    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 04:42:33.491767 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.491722    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 04:42:33.491767 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.491731    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasSufficientPID"
Jun 02 04:42:33.491853 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.491769    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.499528 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.499514    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.499576 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.499534    2577 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-23.ec2.internal\": node \"ip-10-0-135-23.ec2.internal\" not found"
Jun 02 04:42:33.529884 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.529864    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-23.ec2.internal\" not found"
Jun 02 04:42:33.539922 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.539900    2577 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-23.ec2.internal"]
Jun 02 04:42:33.539999 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.539982    2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 04:42:33.540676 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.540662    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 04:42:33.540727 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.540688    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 04:42:33.540727 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.540699    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasSufficientPID"
Jun 02 04:42:33.541853 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.541842    2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 04:42:33.542015 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.542002    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.542054 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.542029    2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 04:42:33.542479 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.542455    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 04:42:33.542479 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.542463    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 04:42:33.542558 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.542485    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 04:42:33.542558 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.542496    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasSufficientPID"
Jun 02 04:42:33.542624 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.542486    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 04:42:33.542658 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.542642    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasSufficientPID"
Jun 02 04:42:33.543602 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.543590    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.543645 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.543613    2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
Jun 02 04:42:33.545823 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.545789    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasSufficientMemory"
Jun 02 04:42:33.545902 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.545833    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasNoDiskPressure"
Jun 02 04:42:33.545902 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.545849    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeHasSufficientPID"
Jun 02 04:42:33.573358 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.573340    2577 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-23.ec2.internal\" not found" node="ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.577533 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.577518    2577 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-23.ec2.internal\" not found" node="ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.627191 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.627173    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a71245ffd49693c937e706ccff6c087-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal\" (UID: \"4a71245ffd49693c937e706ccff6c087\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.627261 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.627195    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8d89a7cb71da8618cf393c332af1d5c1-config\") pod \"kube-apiserver-proxy-ip-10-0-135-23.ec2.internal\" (UID: \"8d89a7cb71da8618cf393c332af1d5c1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.627261 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.627219    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4a71245ffd49693c937e706ccff6c087-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal\" (UID: \"4a71245ffd49693c937e706ccff6c087\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.630281 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.630267    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-23.ec2.internal\" not found"
Jun 02 04:42:33.728030 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.728012    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a71245ffd49693c937e706ccff6c087-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal\" (UID: \"4a71245ffd49693c937e706ccff6c087\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.728096 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.728037    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8d89a7cb71da8618cf393c332af1d5c1-config\") pod \"kube-apiserver-proxy-ip-10-0-135-23.ec2.internal\" (UID: \"8d89a7cb71da8618cf393c332af1d5c1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.728096 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.728055    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4a71245ffd49693c937e706ccff6c087-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal\" (UID: \"4a71245ffd49693c937e706ccff6c087\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.728160 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.728098    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a71245ffd49693c937e706ccff6c087-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal\" (UID: \"4a71245ffd49693c937e706ccff6c087\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.728160 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.728105    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8d89a7cb71da8618cf393c332af1d5c1-config\") pod \"kube-apiserver-proxy-ip-10-0-135-23.ec2.internal\" (UID: \"8d89a7cb71da8618cf393c332af1d5c1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.728160 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.728099    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4a71245ffd49693c937e706ccff6c087-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal\" (UID: \"4a71245ffd49693c937e706ccff6c087\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.731118 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.731102    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-23.ec2.internal\" not found"
Jun 02 04:42:33.831850 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.831791    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-23.ec2.internal\" not found"
Jun 02 04:42:33.875948 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.875932    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.881451 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:33.881436    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-23.ec2.internal"
Jun 02 04:42:33.932420 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:33.932388    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-23.ec2.internal\" not found"
Jun 02 04:42:34.032871 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:34.032841    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-23.ec2.internal\" not found"
Jun 02 04:42:34.133345 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:34.133289    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-23.ec2.internal\" not found"
Jun 02 04:42:34.233858 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:34.233840    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-23.ec2.internal\" not found"
Jun 02 04:42:34.243004 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.242988    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jun 02 04:42:34.243122 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.243107    2577 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jun 02 04:42:34.243159 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.243136    2577 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jun 02 04:42:34.323892 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.323857    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 04:37:33 +0000 UTC" deadline="2027-10-31 00:30:59.691373845 +0000 UTC"
Jun 02 04:42:34.323892 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.323890    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12379h48m25.367486867s"
Jun 02 04:42:34.325737 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.325715    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jun 02 04:42:34.333902 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:34.333883    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-23.ec2.internal\" not found"
Jun 02 04:42:34.335725 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.335707    2577 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jun 02 04:42:34.355397 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:34.355372    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d89a7cb71da8618cf393c332af1d5c1.slice/crio-1113c6bb78ff169a3ac1b8fd69343b128c18fce3a2cb782fa548d088e12e3118 WatchSource:0}: Error finding container 1113c6bb78ff169a3ac1b8fd69343b128c18fce3a2cb782fa548d088e12e3118: Status 404 returned error can't find the container with id 1113c6bb78ff169a3ac1b8fd69343b128c18fce3a2cb782fa548d088e12e3118
Jun 02 04:42:34.355592 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:34.355579    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a71245ffd49693c937e706ccff6c087.slice/crio-b22e2560b6f647fafbf3774961a86cbdad3f5df391187d486d0200b516c65483 WatchSource:0}: Error finding container b22e2560b6f647fafbf3774961a86cbdad3f5df391187d486d0200b516c65483: Status 404 returned error can't find the container with id b22e2560b6f647fafbf3774961a86cbdad3f5df391187d486d0200b516c65483
Jun 02 04:42:34.358781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.358760    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 04:42:34.360793 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.360652    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tcxqd"
Jun 02 04:42:34.367067 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.367051    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tcxqd"
Jun 02 04:42:34.434256 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:34.434207    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-23.ec2.internal\" not found"
Jun 02 04:42:34.442015 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.441971    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-23.ec2.internal" event={"ID":"8d89a7cb71da8618cf393c332af1d5c1","Type":"ContainerStarted","Data":"1113c6bb78ff169a3ac1b8fd69343b128c18fce3a2cb782fa548d088e12e3118"}
Jun 02 04:42:34.442882 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.442863    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal" event={"ID":"4a71245ffd49693c937e706ccff6c087","Type":"ContainerStarted","Data":"b22e2560b6f647fafbf3774961a86cbdad3f5df391187d486d0200b516c65483"}
Jun 02 04:42:34.534394 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:34.534372    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-23.ec2.internal\" not found"
Jun 02 04:42:34.541972 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.541958    2577 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 04:42:34.544313 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.544299    2577 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 04:42:34.626168 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.626146    2577 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal"
Jun 02 04:42:34.640914 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.640896    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jun 02 04:42:34.641579 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.641568    2577 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-23.ec2.internal"
Jun 02 04:42:34.650886 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:34.650863    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jun 02 04:42:35.222514 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.222450    2577 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 04:42:35.305786 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.305758    2577 apiserver.go:52] "Watching apiserver"
Jun 02 04:42:35.313965 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.313943    2577 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jun 02 04:42:35.315526 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.315501    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-qdfbx","kube-system/kube-apiserver-proxy-ip-10-0-135-23.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj","openshift-cluster-node-tuning-operator/tuned-58cf2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal","openshift-multus/multus-additional-cni-plugins-29k6t","openshift-multus/multus-t4m5q","openshift-network-diagnostics/network-check-target-5877c","openshift-image-registry/node-ca-gk5rc","openshift-multus/network-metrics-daemon-jnktq","openshift-network-operator/iptables-alerter-v9tgv","openshift-ovn-kubernetes/ovnkube-node-hgcxc"]
Jun 02 04:42:35.317073 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.317043    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qdfbx"
Jun 02 04:42:35.318114 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.318090    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.319253 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.319228    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.320457 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.320437    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.320612 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.320581    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5j8p2\""
Jun 02 04:42:35.321068 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.320695    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jun 02 04:42:35.321068 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.320587    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4fwb9\""
Jun 02 04:42:35.321068 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.320777    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jun 02 04:42:35.321068 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.320590    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jun 02 04:42:35.321068 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.320971    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jun 02 04:42:35.321068 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.321045    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jun 02 04:42:35.321539 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.321518    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jun 02 04:42:35.321539 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.321537    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jun 02 04:42:35.322011 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.321992    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9xrr7\""
Jun 02 04:42:35.322683 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.322663    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jun 02 04:42:35.322759 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.322745    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.323039 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.323023    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mkxkz\""
Jun 02 04:42:35.323509 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.323369    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jun 02 04:42:35.323509 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.323483    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jun 02 04:42:35.323854 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.323838    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jun 02 04:42:35.323945 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.323842    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jun 02 04:42:35.324216 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.324193    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:35.324294 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:35.324262    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5877c" podUID="f4b0125c-3e37-44fb-a810-b204058d5f45"
Jun 02 04:42:35.325090 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.324930    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jun 02 04:42:35.325336 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.325317    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rhbcw\""
Jun 02 04:42:35.325623 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.325604    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gk5rc"
Jun 02 04:42:35.325909 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.325754    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:35.325909 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:35.325827    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:42:35.326875 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.326855    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v9tgv"
Jun 02 04:42:35.327937 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.327835    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jun 02 04:42:35.327937 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.327840    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jun 02 04:42:35.327937 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.327838    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-c27cb\""
Jun 02 04:42:35.327937 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.327840    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jun 02 04:42:35.328182 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.328157    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.329124 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.329106    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jun 02 04:42:35.329393 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.329377    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jun 02 04:42:35.329470 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.329433    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jun 02 04:42:35.329653 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.329639    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rpwqk\""
Jun 02 04:42:35.330292 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.330273    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jun 02 04:42:35.330727 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.330582    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jun 02 04:42:35.330727 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.330607    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jun 02 04:42:35.330727 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.330608    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jun 02 04:42:35.330969 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.330943    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jun 02 04:42:35.330969 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.330950    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p7whw\""
Jun 02 04:42:35.331704 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.331688    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jun 02 04:42:35.335775 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.335755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-sys\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.335882 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.335783    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-run-multus-certs\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.335882 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.335847    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-sys-fs\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.335986 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.335883    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-sysconfig\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.335986 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.335909    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-sysctl-conf\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.335986 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.335956    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-cnibin\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.336137 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336002    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a77859ea-6f29-411d-b3d1-abc2e33e3745-serviceca\") pod \"node-ca-gk5rc\" (UID: \"a77859ea-6f29-411d-b3d1-abc2e33e3745\") " pod="openshift-image-registry/node-ca-gk5rc"
Jun 02 04:42:35.336137 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336033    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-modprobe-d\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.336137 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336056    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-system-cni-dir\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.336137 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.336137 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336106    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-system-cni-dir\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.336334 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336138    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-etc-selinux\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.336334 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336162    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a77859ea-6f29-411d-b3d1-abc2e33e3745-host\") pod \"node-ca-gk5rc\" (UID: \"a77859ea-6f29-411d-b3d1-abc2e33e3745\") " pod="openshift-image-registry/node-ca-gk5rc"
Jun 02 04:42:35.336334 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336197    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.336334 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336240    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg8fm\" (UniqueName: \"kubernetes.io/projected/f19f54c7-a575-4537-aa41-524e5b5bf483-kube-api-access-fg8fm\") pod \"iptables-alerter-v9tgv\" (UID: \"f19f54c7-a575-4537-aa41-524e5b5bf483\") " pod="openshift-network-operator/iptables-alerter-v9tgv"
Jun 02 04:42:35.336334 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336265    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-run\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.336334 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336288    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-lib-modules\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.336334 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336306    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-tuned\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.336334 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336319    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/260fe733-4ba7-4ed3-ac62-badcb5a113cf-tmp\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.336673 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqgrk\" (UniqueName: \"kubernetes.io/projected/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-kube-api-access-wqgrk\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.336673 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336360    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-multus-conf-dir\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.336673 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336379    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8d6a095d-aaa2-4332-b27e-060a5bc84abd-multus-daemon-config\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.336673 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336399    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj4vs\" (UniqueName: \"kubernetes.io/projected/7f26ffcc-136d-49e4-9072-626c6a31246e-kube-api-access-sj4vs\") pod \"network-metrics-daemon-jnktq\" (UID: \"7f26ffcc-136d-49e4-9072-626c6a31246e\") " pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:35.336673 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336421    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-run-k8s-cni-cncf-io\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.336673 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336458    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-var-lib-cni-bin\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.336673 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336477    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-var-lib-cni-multus\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.336673 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336509    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pc67\" (UniqueName: \"kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67\") pod \"network-check-target-5877c\" (UID: \"f4b0125c-3e37-44fb-a810-b204058d5f45\") " pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:35.336673 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-registration-dir\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.336673 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4ba583d-27eb-4e01-abf6-42dd3767e562-agent-certs\") pod \"konnectivity-agent-qdfbx\" (UID: \"f4ba583d-27eb-4e01-abf6-42dd3767e562\") " pod="kube-system/konnectivity-agent-qdfbx"
Jun 02 04:42:35.336673 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336581    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-run-netns\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.336673 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336614    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms85p\" (UniqueName: \"kubernetes.io/projected/8d6a095d-aaa2-4332-b27e-060a5bc84abd-kube-api-access-ms85p\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.336673 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336652    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4ba583d-27eb-4e01-abf6-42dd3767e562-konnectivity-ca\") pod \"konnectivity-agent-qdfbx\" (UID: \"f4ba583d-27eb-4e01-abf6-42dd3767e562\") " pod="kube-system/konnectivity-agent-qdfbx"
Jun 02 04:42:35.337251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336684    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-sysctl-d\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.337251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-host\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.337251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336745    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-os-release\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.337251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336773    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-os-release\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.337251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336794    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs\") pod \"network-metrics-daemon-jnktq\" (UID: \"7f26ffcc-136d-49e4-9072-626c6a31246e\") " pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:35.337251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c88l5\" (UniqueName: \"kubernetes.io/projected/a77859ea-6f29-411d-b3d1-abc2e33e3745-kube-api-access-c88l5\") pod \"node-ca-gk5rc\" (UID: \"a77859ea-6f29-411d-b3d1-abc2e33e3745\") " pod="openshift-image-registry/node-ca-gk5rc"
Jun 02 04:42:35.337251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336897    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98rsv\" (UniqueName: \"kubernetes.io/projected/260fe733-4ba7-4ed3-ac62-badcb5a113cf-kube-api-access-98rsv\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.337251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336948    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.337251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-kubernetes\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.337251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.336996    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-var-lib-kubelet\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.337251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.337032    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.337251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.337058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-var-lib-kubelet\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.337251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.337090    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-etc-kubernetes\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.337251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.337112    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-socket-dir\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.337867 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.337145    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f19f54c7-a575-4537-aa41-524e5b5bf483-host-slash\") pod \"iptables-alerter-v9tgv\" (UID: \"f19f54c7-a575-4537-aa41-524e5b5bf483\") " pod="openshift-network-operator/iptables-alerter-v9tgv"
Jun 02 04:42:35.337867 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.337474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-cni-binary-copy\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.337867 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.337541    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-device-dir\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.337867 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.337638    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8svt\" (UniqueName: \"kubernetes.io/projected/36ec0192-f588-46d3-bb4d-ee4495c81eeb-kube-api-access-w8svt\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.337867 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.337682    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-multus-cni-dir\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.337867 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.337713    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-hostroot\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.337867 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.337852    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-systemd\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.338180 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.337896    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-cnibin\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.338180 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.337931    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8d6a095d-aaa2-4332-b27e-060a5bc84abd-cni-binary-copy\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.338180 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.337957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-multus-socket-dir-parent\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.338180 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.337994    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f19f54c7-a575-4537-aa41-524e5b5bf483-iptables-alerter-script\") pod \"iptables-alerter-v9tgv\" (UID: \"f19f54c7-a575-4537-aa41-524e5b5bf483\") " pod="openshift-network-operator/iptables-alerter-v9tgv"
Jun 02 04:42:35.367572 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.367546    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 04:37:34 +0000 UTC" deadline="2028-01-24 23:09:55.954769625 +0000 UTC"
Jun 02 04:42:35.367647 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.367572    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14442h27m20.587200907s"
Jun 02 04:42:35.427189 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.427168    2577 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
Jun 02 04:42:35.439182 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439158    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-sysconfig\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.439302 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439187    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-sysctl-conf\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.439302 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439223    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-cnibin\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.439302 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439246    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a77859ea-6f29-411d-b3d1-abc2e33e3745-serviceca\") pod \"node-ca-gk5rc\" (UID: \"a77859ea-6f29-411d-b3d1-abc2e33e3745\") " pod="openshift-image-registry/node-ca-gk5rc"
Jun 02 04:42:35.439302 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439274    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-run-systemd\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.439302 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439288    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-cnibin\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.439302 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-cni-bin\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439320    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-228qh\" (UniqueName: \"kubernetes.io/projected/88d63d7d-55eb-4340-99ae-9c49be3afdbe-kube-api-access-228qh\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439383    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-sysconfig\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439377    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-modprobe-d\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439397    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-sysctl-conf\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439412    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-system-cni-dir\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439428    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439450    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-system-cni-dir\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439466    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-system-cni-dir\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439481    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-modprobe-d\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439495    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-etc-selinux\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439519    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a77859ea-6f29-411d-b3d1-abc2e33e3745-host\") pod \"node-ca-gk5rc\" (UID: \"a77859ea-6f29-411d-b3d1-abc2e33e3745\") " pod="openshift-image-registry/node-ca-gk5rc"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439528    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-system-cni-dir\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439562    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439565    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a77859ea-6f29-411d-b3d1-abc2e33e3745-host\") pod \"node-ca-gk5rc\" (UID: \"a77859ea-6f29-411d-b3d1-abc2e33e3745\") " pod="openshift-image-registry/node-ca-gk5rc"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-node-log\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439611    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-etc-selinux\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.439614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439619    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-log-socket\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439651    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439678    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fg8fm\" (UniqueName: \"kubernetes.io/projected/f19f54c7-a575-4537-aa41-524e5b5bf483-kube-api-access-fg8fm\") pod \"iptables-alerter-v9tgv\" (UID: \"f19f54c7-a575-4537-aa41-524e5b5bf483\") " pod="openshift-network-operator/iptables-alerter-v9tgv"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439698    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439707    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-run-ovn\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439731    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-run\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439754    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-lib-modules\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439778    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-tuned\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439806    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/260fe733-4ba7-4ed3-ac62-badcb5a113cf-tmp\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439850    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wqgrk\" (UniqueName: \"kubernetes.io/projected/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-kube-api-access-wqgrk\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439877    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-multus-conf-dir\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439897    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8d6a095d-aaa2-4332-b27e-060a5bc84abd-multus-daemon-config\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439919    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-lib-modules\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439912    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sj4vs\" (UniqueName: \"kubernetes.io/projected/7f26ffcc-136d-49e4-9072-626c6a31246e-kube-api-access-sj4vs\") pod \"network-metrics-daemon-jnktq\" (UID: \"7f26ffcc-136d-49e4-9072-626c6a31246e\") " pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439990    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-run-k8s-cni-cncf-io\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440018    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-var-lib-cni-bin\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440043    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-var-lib-cni-multus\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.440414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440071    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2pc67\" (UniqueName: \"kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67\") pod \"network-check-target-5877c\" (UID: \"f4b0125c-3e37-44fb-a810-b204058d5f45\") " pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440097    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-registration-dir\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440121    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4ba583d-27eb-4e01-abf6-42dd3767e562-agent-certs\") pod \"konnectivity-agent-qdfbx\" (UID: \"f4ba583d-27eb-4e01-abf6-42dd3767e562\") " pod="kube-system/konnectivity-agent-qdfbx"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-var-lib-openvswitch\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/88d63d7d-55eb-4340-99ae-9c49be3afdbe-env-overrides\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440205    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-run-netns\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440230    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ms85p\" (UniqueName: \"kubernetes.io/projected/8d6a095d-aaa2-4332-b27e-060a5bc84abd-kube-api-access-ms85p\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440254    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4ba583d-27eb-4e01-abf6-42dd3767e562-konnectivity-ca\") pod \"konnectivity-agent-qdfbx\" (UID: \"f4ba583d-27eb-4e01-abf6-42dd3767e562\") " pod="kube-system/konnectivity-agent-qdfbx"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440279    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-sysctl-d\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.439820    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-run\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/88d63d7d-55eb-4340-99ae-9c49be3afdbe-ovnkube-script-lib\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440355    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-host\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440365    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-multus-conf-dir\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440379    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-os-release\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440405    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-os-release\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440416    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-run-k8s-cni-cncf-io\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440431    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs\") pod \"network-metrics-daemon-jnktq\" (UID: \"7f26ffcc-136d-49e4-9072-626c6a31246e\") " pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:35.441200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440458    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-var-lib-cni-bin\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440463    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-systemd-units\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440491    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-etc-openvswitch\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440496    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-var-lib-cni-multus\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440520    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c88l5\" (UniqueName: \"kubernetes.io/projected/a77859ea-6f29-411d-b3d1-abc2e33e3745-kube-api-access-c88l5\") pod \"node-ca-gk5rc\" (UID: \"a77859ea-6f29-411d-b3d1-abc2e33e3745\") " pod="openshift-image-registry/node-ca-gk5rc"
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440553    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-98rsv\" (UniqueName: \"kubernetes.io/projected/260fe733-4ba7-4ed3-ac62-badcb5a113cf-kube-api-access-98rsv\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440569    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440580    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440619    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-kubelet\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440636    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8d6a095d-aaa2-4332-b27e-060a5bc84abd-multus-daemon-config\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440703    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-run-netns\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440707    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440725    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-registration-dir\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.440801    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-os-release\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:35.440843    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:35.440915    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs podName:7f26ffcc-136d-49e4-9072-626c6a31246e nodeName:}" failed. No retries permitted until 2026-06-02 04:42:35.94089309 +0000 UTC m=+2.981824054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs") pod "network-metrics-daemon-jnktq" (UID: "7f26ffcc-136d-49e4-9072-626c6a31246e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 04:42:35.441979 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441279    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441318    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-sysctl-d\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441338    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-host\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441376    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-os-release\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441402    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-kubernetes\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441422    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4ba583d-27eb-4e01-abf6-42dd3767e562-konnectivity-ca\") pod \"konnectivity-agent-qdfbx\" (UID: \"f4ba583d-27eb-4e01-abf6-42dd3767e562\") " pod="kube-system/konnectivity-agent-qdfbx"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441429    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-var-lib-kubelet\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441459    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441488    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-var-lib-kubelet\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441513    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-etc-kubernetes\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441540    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-socket-dir\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441570    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f19f54c7-a575-4537-aa41-524e5b5bf483-host-slash\") pod \"iptables-alerter-v9tgv\" (UID: \"f19f54c7-a575-4537-aa41-524e5b5bf483\") " pod="openshift-network-operator/iptables-alerter-v9tgv"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441646    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-var-lib-kubelet\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441652    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-run-netns\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441679    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-etc-kubernetes\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441689    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-cni-binary-copy\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441697    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-var-lib-kubelet\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.442781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441715    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f19f54c7-a575-4537-aa41-524e5b5bf483-host-slash\") pod \"iptables-alerter-v9tgv\" (UID: \"f19f54c7-a575-4537-aa41-524e5b5bf483\") " pod="openshift-network-operator/iptables-alerter-v9tgv"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441717    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-device-dir\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441779    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-kubernetes\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441793    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-socket-dir\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441834    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w8svt\" (UniqueName: \"kubernetes.io/projected/36ec0192-f588-46d3-bb4d-ee4495c81eeb-kube-api-access-w8svt\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-cni-netd\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441893    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-multus-cni-dir\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441921    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-hostroot\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441967    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/88d63d7d-55eb-4340-99ae-9c49be3afdbe-ovn-node-metrics-cert\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.441994    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-systemd\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442019    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-cnibin\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442046    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8d6a095d-aaa2-4332-b27e-060a5bc84abd-cni-binary-copy\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442082    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442106    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-multus-socket-dir-parent\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442139    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f19f54c7-a575-4537-aa41-524e5b5bf483-iptables-alerter-script\") pod \"iptables-alerter-v9tgv\" (UID: \"f19f54c7-a575-4537-aa41-524e5b5bf483\") " pod="openshift-network-operator/iptables-alerter-v9tgv"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442149    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-multus-cni-dir\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442173    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-slash\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.443531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442204    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-device-dir\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.444240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-run-openvswitch\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.444240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/88d63d7d-55eb-4340-99ae-9c49be3afdbe-ovnkube-config\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.444240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442294    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-sys\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.444240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442326    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-run-multus-certs\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.444240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442382    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-sys-fs\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.444240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442484    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/36ec0192-f588-46d3-bb4d-ee4495c81eeb-sys-fs\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.444240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442479    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8d6a095d-aaa2-4332-b27e-060a5bc84abd-cni-binary-copy\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.444240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442560    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-cnibin\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.444240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442587    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-host-run-multus-certs\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.444240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442629    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-hostroot\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.444240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442674    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-systemd\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.444240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442687    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a77859ea-6f29-411d-b3d1-abc2e33e3745-serviceca\") pod \"node-ca-gk5rc\" (UID: \"a77859ea-6f29-411d-b3d1-abc2e33e3745\") " pod="openshift-image-registry/node-ca-gk5rc"
Jun 02 04:42:35.444240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442714    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8d6a095d-aaa2-4332-b27e-060a5bc84abd-multus-socket-dir-parent\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.444240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.442723    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/260fe733-4ba7-4ed3-ac62-badcb5a113cf-sys\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.444240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.443137    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f19f54c7-a575-4537-aa41-524e5b5bf483-iptables-alerter-script\") pod \"iptables-alerter-v9tgv\" (UID: \"f19f54c7-a575-4537-aa41-524e5b5bf483\") " pod="openshift-network-operator/iptables-alerter-v9tgv"
Jun 02 04:42:35.444240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.443215    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-cni-binary-copy\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.445059 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.445039    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/260fe733-4ba7-4ed3-ac62-badcb5a113cf-etc-tuned\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.445133 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.445069    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/260fe733-4ba7-4ed3-ac62-badcb5a113cf-tmp\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.445316 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.445292    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4ba583d-27eb-4e01-abf6-42dd3767e562-agent-certs\") pod \"konnectivity-agent-qdfbx\" (UID: \"f4ba583d-27eb-4e01-abf6-42dd3767e562\") " pod="kube-system/konnectivity-agent-qdfbx"
Jun 02 04:42:35.450308 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:35.450013    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 04:42:35.450308 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:35.450033    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 04:42:35.450308 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:35.450045    2577 projected.go:194] Error preparing data for projected volume kube-api-access-2pc67 for pod openshift-network-diagnostics/network-check-target-5877c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 04:42:35.450308 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:35.450096    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67 podName:f4b0125c-3e37-44fb-a810-b204058d5f45 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:35.950079192 +0000 UTC m=+2.991010157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2pc67" (UniqueName: "kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67") pod "network-check-target-5877c" (UID: "f4b0125c-3e37-44fb-a810-b204058d5f45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 04:42:35.451473 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.450932    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms85p\" (UniqueName: \"kubernetes.io/projected/8d6a095d-aaa2-4332-b27e-060a5bc84abd-kube-api-access-ms85p\") pod \"multus-t4m5q\" (UID: \"8d6a095d-aaa2-4332-b27e-060a5bc84abd\") " pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.452317 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.452290    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj4vs\" (UniqueName: \"kubernetes.io/projected/7f26ffcc-136d-49e4-9072-626c6a31246e-kube-api-access-sj4vs\") pod \"network-metrics-daemon-jnktq\" (UID: \"7f26ffcc-136d-49e4-9072-626c6a31246e\") " pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:35.452679 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.452659    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98rsv\" (UniqueName: \"kubernetes.io/projected/260fe733-4ba7-4ed3-ac62-badcb5a113cf-kube-api-access-98rsv\") pod \"tuned-58cf2\" (UID: \"260fe733-4ba7-4ed3-ac62-badcb5a113cf\") " pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.453252 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.453195    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqgrk\" (UniqueName: \"kubernetes.io/projected/dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd-kube-api-access-wqgrk\") pod \"multus-additional-cni-plugins-29k6t\" (UID: \"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd\") " pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.455763 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.453268    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg8fm\" (UniqueName: \"kubernetes.io/projected/f19f54c7-a575-4537-aa41-524e5b5bf483-kube-api-access-fg8fm\") pod \"iptables-alerter-v9tgv\" (UID: \"f19f54c7-a575-4537-aa41-524e5b5bf483\") " pod="openshift-network-operator/iptables-alerter-v9tgv"
Jun 02 04:42:35.456399 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.456375    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8svt\" (UniqueName: \"kubernetes.io/projected/36ec0192-f588-46d3-bb4d-ee4495c81eeb-kube-api-access-w8svt\") pod \"aws-ebs-csi-driver-node-8ktnj\" (UID: \"36ec0192-f588-46d3-bb4d-ee4495c81eeb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.456638 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.456608    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c88l5\" (UniqueName: \"kubernetes.io/projected/a77859ea-6f29-411d-b3d1-abc2e33e3745-kube-api-access-c88l5\") pod \"node-ca-gk5rc\" (UID: \"a77859ea-6f29-411d-b3d1-abc2e33e3745\") " pod="openshift-image-registry/node-ca-gk5rc"
Jun 02 04:42:35.543079 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543055    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-systemd-units\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543079 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543084    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-etc-openvswitch\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543279 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543102    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-kubelet\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543279 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543124    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543279 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543152    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543279 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543166    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-etc-openvswitch\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543279 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543172    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-systemd-units\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543279 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543181    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-kubelet\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543279 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543181    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-run-netns\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543279 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543201    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543279 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543218    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-run-netns\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543279 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543224    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543279 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543237    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-cni-netd\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543279 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543259    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/88d63d7d-55eb-4340-99ae-9c49be3afdbe-ovn-node-metrics-cert\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543279 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543278    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-slash\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543301    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-run-openvswitch\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543325    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/88d63d7d-55eb-4340-99ae-9c49be3afdbe-ovnkube-config\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543353    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-run-openvswitch\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543357    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-run-systemd\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543321    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-cni-netd\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543414    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-cni-bin\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543390    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-run-systemd\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543389    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-cni-bin\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543438    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-228qh\" (UniqueName: \"kubernetes.io/projected/88d63d7d-55eb-4340-99ae-9c49be3afdbe-kube-api-access-228qh\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543463    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-node-log\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543470    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-host-slash\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543486    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-log-socket\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543517    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-run-ovn\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543521    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-node-log\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543557    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-log-socket\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543567    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-var-lib-openvswitch\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543591    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/88d63d7d-55eb-4340-99ae-9c49be3afdbe-env-overrides\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.543850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543597    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-var-lib-openvswitch\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.544397 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543606    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/88d63d7d-55eb-4340-99ae-9c49be3afdbe-run-ovn\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.544397 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543618    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/88d63d7d-55eb-4340-99ae-9c49be3afdbe-ovnkube-script-lib\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.544397 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543965    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/88d63d7d-55eb-4340-99ae-9c49be3afdbe-ovnkube-config\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.544397 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.543998    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/88d63d7d-55eb-4340-99ae-9c49be3afdbe-env-overrides\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.544397 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.544061    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/88d63d7d-55eb-4340-99ae-9c49be3afdbe-ovnkube-script-lib\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.545249 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.545225    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/88d63d7d-55eb-4340-99ae-9c49be3afdbe-ovn-node-metrics-cert\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.552042 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.552021    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-228qh\" (UniqueName: \"kubernetes.io/projected/88d63d7d-55eb-4340-99ae-9c49be3afdbe-kube-api-access-228qh\") pod \"ovnkube-node-hgcxc\" (UID: \"88d63d7d-55eb-4340-99ae-9c49be3afdbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.629027 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.628998    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qdfbx"
Jun 02 04:42:35.636725 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.636702    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj"
Jun 02 04:42:35.645436 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.645417    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-58cf2"
Jun 02 04:42:35.650978 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.650960    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-29k6t"
Jun 02 04:42:35.659146 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.659124    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t4m5q"
Jun 02 04:42:35.664678 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.664660    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gk5rc"
Jun 02 04:42:35.672169 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.672153    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v9tgv"
Jun 02 04:42:35.675789 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.675770    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:35.805562 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.805483    2577 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jun 02 04:42:35.872055 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:35.871868    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88d63d7d_55eb_4340_99ae_9c49be3afdbe.slice/crio-5894693b785272fd0c318f27f8fc53a206dbbe671fd01431093ef0768be19255 WatchSource:0}: Error finding container 5894693b785272fd0c318f27f8fc53a206dbbe671fd01431093ef0768be19255: Status 404 returned error can't find the container with id 5894693b785272fd0c318f27f8fc53a206dbbe671fd01431093ef0768be19255
Jun 02 04:42:35.873050 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:35.873023    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36ec0192_f588_46d3_bb4d_ee4495c81eeb.slice/crio-21ce0fe86ed906f59c392fb2553044d3f6e5916a87355435d0d11096561a3fb2 WatchSource:0}: Error finding container 21ce0fe86ed906f59c392fb2553044d3f6e5916a87355435d0d11096561a3fb2: Status 404 returned error can't find the container with id 21ce0fe86ed906f59c392fb2553044d3f6e5916a87355435d0d11096561a3fb2
Jun 02 04:42:35.875990 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:35.875970    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4ba583d_27eb_4e01_abf6_42dd3767e562.slice/crio-9d9bf358d9ffa0068df2226220628aa080fd26a7ec9a025f2608a77c46309843 WatchSource:0}: Error finding container 9d9bf358d9ffa0068df2226220628aa080fd26a7ec9a025f2608a77c46309843: Status 404 returned error can't find the container with id 9d9bf358d9ffa0068df2226220628aa080fd26a7ec9a025f2608a77c46309843
Jun 02 04:42:35.878515 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:35.877931    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd6cf343_d9aa_452b_9bc8_4f7fc753b6dd.slice/crio-4d119269707f46f47e51007988b416f41a324c9209bebf34d568f168e0858515 WatchSource:0}: Error finding container 4d119269707f46f47e51007988b416f41a324c9209bebf34d568f168e0858515: Status 404 returned error can't find the container with id 4d119269707f46f47e51007988b416f41a324c9209bebf34d568f168e0858515
Jun 02 04:42:35.879883 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:35.879840    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod260fe733_4ba7_4ed3_ac62_badcb5a113cf.slice/crio-8db8776cd0bcf7feacf0561f292943491b5a7f9826b666bbac54e19f12a39bb6 WatchSource:0}: Error finding container 8db8776cd0bcf7feacf0561f292943491b5a7f9826b666bbac54e19f12a39bb6: Status 404 returned error can't find the container with id 8db8776cd0bcf7feacf0561f292943491b5a7f9826b666bbac54e19f12a39bb6
Jun 02 04:42:35.881193 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:35.881169    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda77859ea_6f29_411d_b3d1_abc2e33e3745.slice/crio-075c381dde43af1464d499004d9b08dd47de5f8fe3901ecb0c8dcefcd7874104 WatchSource:0}: Error finding container 075c381dde43af1464d499004d9b08dd47de5f8fe3901ecb0c8dcefcd7874104: Status 404 returned error can't find the container with id 075c381dde43af1464d499004d9b08dd47de5f8fe3901ecb0c8dcefcd7874104
Jun 02 04:42:35.882162 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:35.882132    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf19f54c7_a575_4537_aa41_524e5b5bf483.slice/crio-c9ec513b7009af979fa8bd77884515426e51c0c7bd3f634c981c9a693ca4e290 WatchSource:0}: Error finding container c9ec513b7009af979fa8bd77884515426e51c0c7bd3f634c981c9a693ca4e290: Status 404 returned error can't find the container with id c9ec513b7009af979fa8bd77884515426e51c0c7bd3f634c981c9a693ca4e290
Jun 02 04:42:35.883099 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:35.883068    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d6a095d_aaa2_4332_b27e_060a5bc84abd.slice/crio-f584a565e7f46c0b934f0506f796e3a22d7eee77b80db5a54679352baccfbcb8 WatchSource:0}: Error finding container f584a565e7f46c0b934f0506f796e3a22d7eee77b80db5a54679352baccfbcb8: Status 404 returned error can't find the container with id f584a565e7f46c0b934f0506f796e3a22d7eee77b80db5a54679352baccfbcb8
Jun 02 04:42:35.947712 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:35.947592    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs\") pod \"network-metrics-daemon-jnktq\" (UID: \"7f26ffcc-136d-49e4-9072-626c6a31246e\") " pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:35.947831 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:35.947694    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 04:42:35.947831 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:35.947799    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs podName:7f26ffcc-136d-49e4-9072-626c6a31246e nodeName:}" failed. No retries permitted until 2026-06-02 04:42:36.947778058 +0000 UTC m=+3.988709020 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs") pod "network-metrics-daemon-jnktq" (UID: "7f26ffcc-136d-49e4-9072-626c6a31246e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 04:42:36.048894 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:36.048870    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2pc67\" (UniqueName: \"kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67\") pod \"network-check-target-5877c\" (UID: \"f4b0125c-3e37-44fb-a810-b204058d5f45\") " pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:36.049079 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:36.049050    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 04:42:36.049079 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:36.049075    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 04:42:36.049195 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:36.049088    2577 projected.go:194] Error preparing data for projected volume kube-api-access-2pc67 for pod openshift-network-diagnostics/network-check-target-5877c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 04:42:36.049195 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:36.049148    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67 podName:f4b0125c-3e37-44fb-a810-b204058d5f45 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:37.049127941 +0000 UTC m=+4.090058892 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pc67" (UniqueName: "kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67") pod "network-check-target-5877c" (UID: "f4b0125c-3e37-44fb-a810-b204058d5f45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 04:42:36.368407 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:36.368329    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 04:37:34 +0000 UTC" deadline="2028-01-08 09:04:30.228351435 +0000 UTC"
Jun 02 04:42:36.368407 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:36.368364    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14044h21m53.859990803s"
Jun 02 04:42:36.439714 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:36.439684    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:36.439896 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:36.439837    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:42:36.454796 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:36.454679    2577 generic.go:358] "Generic (PLEG): container finished" podID="4a71245ffd49693c937e706ccff6c087" containerID="96960c20a0cec49830f797259b8d08fc9b0294f511dae8d12de586549e4a7ead" exitCode=0
Jun 02 04:42:36.454796 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:36.454759    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal" event={"ID":"4a71245ffd49693c937e706ccff6c087","Type":"ContainerDied","Data":"96960c20a0cec49830f797259b8d08fc9b0294f511dae8d12de586549e4a7ead"}
Jun 02 04:42:36.459264 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:36.459237    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gk5rc" event={"ID":"a77859ea-6f29-411d-b3d1-abc2e33e3745","Type":"ContainerStarted","Data":"075c381dde43af1464d499004d9b08dd47de5f8fe3901ecb0c8dcefcd7874104"}
Jun 02 04:42:36.466086 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:36.466054    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29k6t" event={"ID":"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd","Type":"ContainerStarted","Data":"4d119269707f46f47e51007988b416f41a324c9209bebf34d568f168e0858515"}
Jun 02 04:42:36.469798 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:36.469761    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qdfbx" event={"ID":"f4ba583d-27eb-4e01-abf6-42dd3767e562","Type":"ContainerStarted","Data":"9d9bf358d9ffa0068df2226220628aa080fd26a7ec9a025f2608a77c46309843"}
Jun 02 04:42:36.476971 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:36.476943    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj" event={"ID":"36ec0192-f588-46d3-bb4d-ee4495c81eeb","Type":"ContainerStarted","Data":"21ce0fe86ed906f59c392fb2553044d3f6e5916a87355435d0d11096561a3fb2"}
Jun 02 04:42:36.479481 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:36.479401    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc" event={"ID":"88d63d7d-55eb-4340-99ae-9c49be3afdbe","Type":"ContainerStarted","Data":"5894693b785272fd0c318f27f8fc53a206dbbe671fd01431093ef0768be19255"}
Jun 02 04:42:36.484485 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:36.484461    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-23.ec2.internal" event={"ID":"8d89a7cb71da8618cf393c332af1d5c1","Type":"ContainerStarted","Data":"136e6ed272b00738515397e58238cc9ee2273ed5fb33a693a910fc378db6a029"}
Jun 02 04:42:36.487695 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:36.487672    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t4m5q" event={"ID":"8d6a095d-aaa2-4332-b27e-060a5bc84abd","Type":"ContainerStarted","Data":"f584a565e7f46c0b934f0506f796e3a22d7eee77b80db5a54679352baccfbcb8"}
Jun 02 04:42:36.491985 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:36.491920    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v9tgv" event={"ID":"f19f54c7-a575-4537-aa41-524e5b5bf483","Type":"ContainerStarted","Data":"c9ec513b7009af979fa8bd77884515426e51c0c7bd3f634c981c9a693ca4e290"}
Jun 02 04:42:36.497103 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:36.497048    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-58cf2" event={"ID":"260fe733-4ba7-4ed3-ac62-badcb5a113cf","Type":"ContainerStarted","Data":"8db8776cd0bcf7feacf0561f292943491b5a7f9826b666bbac54e19f12a39bb6"}
Jun 02 04:42:36.957135 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:36.957046    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs\") pod \"network-metrics-daemon-jnktq\" (UID: \"7f26ffcc-136d-49e4-9072-626c6a31246e\") " pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:36.957284 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:36.957213    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 04:42:36.957284 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:36.957272    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs podName:7f26ffcc-136d-49e4-9072-626c6a31246e nodeName:}" failed. No retries permitted until 2026-06-02 04:42:38.95725467 +0000 UTC m=+5.998185635 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs") pod "network-metrics-daemon-jnktq" (UID: "7f26ffcc-136d-49e4-9072-626c6a31246e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 04:42:37.058450 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:37.057845    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2pc67\" (UniqueName: \"kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67\") pod \"network-check-target-5877c\" (UID: \"f4b0125c-3e37-44fb-a810-b204058d5f45\") " pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:37.058450 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:37.058027    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 04:42:37.058450 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:37.058045    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 04:42:37.058450 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:37.058058    2577 projected.go:194] Error preparing data for projected volume kube-api-access-2pc67 for pod openshift-network-diagnostics/network-check-target-5877c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 04:42:37.058450 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:37.058113    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67 podName:f4b0125c-3e37-44fb-a810-b204058d5f45 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:39.058094679 +0000 UTC m=+6.099025632 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pc67" (UniqueName: "kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67") pod "network-check-target-5877c" (UID: "f4b0125c-3e37-44fb-a810-b204058d5f45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 04:42:37.442072 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:37.442042    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:37.442523 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:37.442161    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5877c" podUID="f4b0125c-3e37-44fb-a810-b204058d5f45"
Jun 02 04:42:37.504789 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:37.504752    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal" event={"ID":"4a71245ffd49693c937e706ccff6c087","Type":"ContainerStarted","Data":"e8343d44a0ed8972679beff79cdec06974f01d74fab9679f58cfd6582c06a9ef"}
Jun 02 04:42:37.521531 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:37.520293    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-23.ec2.internal" podStartSLOduration=3.520275275 podStartE2EDuration="3.520275275s" podCreationTimestamp="2026-06-02 04:42:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 04:42:36.500436372 +0000 UTC m=+3.541367348" watchObservedRunningTime="2026-06-02 04:42:37.520275275 +0000 UTC m=+4.561206249"
Jun 02 04:42:38.441663 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:38.441147    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:38.441663 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:38.441283    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:42:38.971829 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:38.971782    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs\") pod \"network-metrics-daemon-jnktq\" (UID: \"7f26ffcc-136d-49e4-9072-626c6a31246e\") " pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:38.972258 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:38.971936    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 04:42:38.972258 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:38.972008    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs podName:7f26ffcc-136d-49e4-9072-626c6a31246e nodeName:}" failed. No retries permitted until 2026-06-02 04:42:42.971986853 +0000 UTC m=+10.012917821 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs") pod "network-metrics-daemon-jnktq" (UID: "7f26ffcc-136d-49e4-9072-626c6a31246e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 04:42:39.072705 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:39.072645    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2pc67\" (UniqueName: \"kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67\") pod \"network-check-target-5877c\" (UID: \"f4b0125c-3e37-44fb-a810-b204058d5f45\") " pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:39.072910 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:39.072830    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 04:42:39.072910 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:39.072852    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 04:42:39.072910 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:39.072866    2577 projected.go:194] Error preparing data for projected volume kube-api-access-2pc67 for pod openshift-network-diagnostics/network-check-target-5877c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 04:42:39.073092 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:39.072923    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67 podName:f4b0125c-3e37-44fb-a810-b204058d5f45 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:43.072903209 +0000 UTC m=+10.113834165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pc67" (UniqueName: "kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67") pod "network-check-target-5877c" (UID: "f4b0125c-3e37-44fb-a810-b204058d5f45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 04:42:39.439633 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:39.439600    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:39.439857 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:39.439736    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5877c" podUID="f4b0125c-3e37-44fb-a810-b204058d5f45"
Jun 02 04:42:40.439416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:40.439385    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:40.440018 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:40.439532    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:42:41.439892 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.439862    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:41.440312 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:41.439993    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5877c" podUID="f4b0125c-3e37-44fb-a810-b204058d5f45"
Jun 02 04:42:41.540010 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.539694    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-23.ec2.internal" podStartSLOduration=7.539674449 podStartE2EDuration="7.539674449s" podCreationTimestamp="2026-06-02 04:42:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 04:42:37.521516442 +0000 UTC m=+4.562447417" watchObservedRunningTime="2026-06-02 04:42:41.539674449 +0000 UTC m=+8.580605423"
Jun 02 04:42:41.540843 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.540202    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-vfq72"]
Jun 02 04:42:41.544483 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.544457    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vfq72"
Jun 02 04:42:41.550453 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.550085    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jun 02 04:42:41.550453 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.550308    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jun 02 04:42:41.552375 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.552286    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zrgp6\""
Jun 02 04:42:41.592686 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.592659    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9rld\" (UniqueName: \"kubernetes.io/projected/f99eac06-93b0-4b1b-aaa0-285d781e5412-kube-api-access-w9rld\") pod \"node-resolver-vfq72\" (UID: \"f99eac06-93b0-4b1b-aaa0-285d781e5412\") " pod="openshift-dns/node-resolver-vfq72"
Jun 02 04:42:41.592778 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.592726    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f99eac06-93b0-4b1b-aaa0-285d781e5412-tmp-dir\") pod \"node-resolver-vfq72\" (UID: \"f99eac06-93b0-4b1b-aaa0-285d781e5412\") " pod="openshift-dns/node-resolver-vfq72"
Jun 02 04:42:41.592778 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.592759    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f99eac06-93b0-4b1b-aaa0-285d781e5412-hosts-file\") pod \"node-resolver-vfq72\" (UID: \"f99eac06-93b0-4b1b-aaa0-285d781e5412\") " pod="openshift-dns/node-resolver-vfq72"
Jun 02 04:42:41.693627 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.693331    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f99eac06-93b0-4b1b-aaa0-285d781e5412-tmp-dir\") pod \"node-resolver-vfq72\" (UID: \"f99eac06-93b0-4b1b-aaa0-285d781e5412\") " pod="openshift-dns/node-resolver-vfq72"
Jun 02 04:42:41.693627 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.693386    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f99eac06-93b0-4b1b-aaa0-285d781e5412-hosts-file\") pod \"node-resolver-vfq72\" (UID: \"f99eac06-93b0-4b1b-aaa0-285d781e5412\") " pod="openshift-dns/node-resolver-vfq72"
Jun 02 04:42:41.693627 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.693457    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9rld\" (UniqueName: \"kubernetes.io/projected/f99eac06-93b0-4b1b-aaa0-285d781e5412-kube-api-access-w9rld\") pod \"node-resolver-vfq72\" (UID: \"f99eac06-93b0-4b1b-aaa0-285d781e5412\") " pod="openshift-dns/node-resolver-vfq72"
Jun 02 04:42:41.693627 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.693551    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f99eac06-93b0-4b1b-aaa0-285d781e5412-hosts-file\") pod \"node-resolver-vfq72\" (UID: \"f99eac06-93b0-4b1b-aaa0-285d781e5412\") " pod="openshift-dns/node-resolver-vfq72"
Jun 02 04:42:41.693976 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.693729    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f99eac06-93b0-4b1b-aaa0-285d781e5412-tmp-dir\") pod \"node-resolver-vfq72\" (UID: \"f99eac06-93b0-4b1b-aaa0-285d781e5412\") " pod="openshift-dns/node-resolver-vfq72"
Jun 02 04:42:41.703411 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.703363    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9rld\" (UniqueName: \"kubernetes.io/projected/f99eac06-93b0-4b1b-aaa0-285d781e5412-kube-api-access-w9rld\") pod \"node-resolver-vfq72\" (UID: \"f99eac06-93b0-4b1b-aaa0-285d781e5412\") " pod="openshift-dns/node-resolver-vfq72"
Jun 02 04:42:41.860340 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:41.860301    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vfq72"
Jun 02 04:42:42.439974 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:42.439543    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:42.439974 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:42.439688    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:42:43.004515 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:43.004321    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs\") pod \"network-metrics-daemon-jnktq\" (UID: \"7f26ffcc-136d-49e4-9072-626c6a31246e\") " pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:43.004515 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:43.004464    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 04:42:43.004515 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:43.004537    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs podName:7f26ffcc-136d-49e4-9072-626c6a31246e nodeName:}" failed. No retries permitted until 2026-06-02 04:42:51.004516268 +0000 UTC m=+18.045447219 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs") pod "network-metrics-daemon-jnktq" (UID: "7f26ffcc-136d-49e4-9072-626c6a31246e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 04:42:43.105869 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:43.105506    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2pc67\" (UniqueName: \"kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67\") pod \"network-check-target-5877c\" (UID: \"f4b0125c-3e37-44fb-a810-b204058d5f45\") " pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:43.105869 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:43.105652    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 04:42:43.105869 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:43.105676    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 04:42:43.105869 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:43.105717    2577 projected.go:194] Error preparing data for projected volume kube-api-access-2pc67 for pod openshift-network-diagnostics/network-check-target-5877c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 04:42:43.105869 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:43.105799    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67 podName:f4b0125c-3e37-44fb-a810-b204058d5f45 nodeName:}" failed. No retries permitted until 2026-06-02 04:42:51.105781241 +0000 UTC m=+18.146712193 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pc67" (UniqueName: "kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67") pod "network-check-target-5877c" (UID: "f4b0125c-3e37-44fb-a810-b204058d5f45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 04:42:43.441599 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:43.440385    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:43.441599 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:43.440490    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5877c" podUID="f4b0125c-3e37-44fb-a810-b204058d5f45"
Jun 02 04:42:44.439977 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:44.439912    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:44.440134 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:44.440042    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:42:45.439465 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:45.439433    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:45.439847 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:45.439562    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5877c" podUID="f4b0125c-3e37-44fb-a810-b204058d5f45"
Jun 02 04:42:46.439648 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:46.439608    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:46.440153 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:46.439714    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:42:47.439623 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:47.439594    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:47.439793 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:47.439713    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5877c" podUID="f4b0125c-3e37-44fb-a810-b204058d5f45"
Jun 02 04:42:48.440127 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:48.440093    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:48.440505 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:48.440206    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:42:49.440038 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:49.440005    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:49.440196 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:49.440129    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5877c" podUID="f4b0125c-3e37-44fb-a810-b204058d5f45"
Jun 02 04:42:50.439784 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:50.439752    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:50.439991 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:50.439910    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:42:51.063655 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:51.063618    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs\") pod \"network-metrics-daemon-jnktq\" (UID: \"7f26ffcc-136d-49e4-9072-626c6a31246e\") " pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:51.064076 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:51.063751    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 04:42:51.064076 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:51.063836    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs podName:7f26ffcc-136d-49e4-9072-626c6a31246e nodeName:}" failed. No retries permitted until 2026-06-02 04:43:07.063802656 +0000 UTC m=+34.104733627 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs") pod "network-metrics-daemon-jnktq" (UID: "7f26ffcc-136d-49e4-9072-626c6a31246e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jun 02 04:42:51.164864 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:51.164828    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2pc67\" (UniqueName: \"kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67\") pod \"network-check-target-5877c\" (UID: \"f4b0125c-3e37-44fb-a810-b204058d5f45\") " pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:51.164996 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:51.164927    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jun 02 04:42:51.164996 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:51.164943    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jun 02 04:42:51.164996 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:51.164952    2577 projected.go:194] Error preparing data for projected volume kube-api-access-2pc67 for pod openshift-network-diagnostics/network-check-target-5877c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 04:42:51.165146 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:51.165007    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67 podName:f4b0125c-3e37-44fb-a810-b204058d5f45 nodeName:}" failed. No retries permitted until 2026-06-02 04:43:07.164988312 +0000 UTC m=+34.205919268 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pc67" (UniqueName: "kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67") pod "network-check-target-5877c" (UID: "f4b0125c-3e37-44fb-a810-b204058d5f45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jun 02 04:42:51.439919 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:51.439843    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:51.440065 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:51.439946    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5877c" podUID="f4b0125c-3e37-44fb-a810-b204058d5f45"
Jun 02 04:42:52.440294 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:52.440260    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:52.440660 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:52.440381    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:42:53.440321 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:53.440294    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:53.440767 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:53.440385    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5877c" podUID="f4b0125c-3e37-44fb-a810-b204058d5f45"
Jun 02 04:42:53.615386 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:42:53.615361    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99eac06_93b0_4b1b_aaa0_285d781e5412.slice/crio-0cff9c1ee54ab73d28b87d444327537ecd9f49d37a3f70f9e523db5823c31833 WatchSource:0}: Error finding container 0cff9c1ee54ab73d28b87d444327537ecd9f49d37a3f70f9e523db5823c31833: Status 404 returned error can't find the container with id 0cff9c1ee54ab73d28b87d444327537ecd9f49d37a3f70f9e523db5823c31833
Jun 02 04:42:54.439630 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.439429    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:54.439754 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:54.439735    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:42:54.533213 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.533180    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t4m5q" event={"ID":"8d6a095d-aaa2-4332-b27e-060a5bc84abd","Type":"ContainerStarted","Data":"28d5c1e5af9b8c6a7e80e93dd06bbe22f2c59b37ecb2e96b4d73a80a0ac7cbc3"}
Jun 02 04:42:54.534847 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.534802    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-58cf2" event={"ID":"260fe733-4ba7-4ed3-ac62-badcb5a113cf","Type":"ContainerStarted","Data":"68d55a131c742470016700c3f7c60859b05237c146a97d19756c99b9c7a4e9ef"}
Jun 02 04:42:54.536286 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.536259    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gk5rc" event={"ID":"a77859ea-6f29-411d-b3d1-abc2e33e3745","Type":"ContainerStarted","Data":"3dac468abf22c9f0b9db82d085bd98601b75dd1e6834bb125430a98dca22bfd6"}
Jun 02 04:42:54.537745 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.537723    2577 generic.go:358] "Generic (PLEG): container finished" podID="dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd" containerID="ca48fe5abd44664e8ada0a912b1a8802643d954374e632b49ec0aad2184f6477" exitCode=0
Jun 02 04:42:54.537870 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.537782    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29k6t" event={"ID":"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd","Type":"ContainerDied","Data":"ca48fe5abd44664e8ada0a912b1a8802643d954374e632b49ec0aad2184f6477"}
Jun 02 04:42:54.539399 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.539162    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qdfbx" event={"ID":"f4ba583d-27eb-4e01-abf6-42dd3767e562","Type":"ContainerStarted","Data":"41c07e679b0bd9e3dcba6b05f1d37671b0102c8c45850b628e2efe6a01ec5b3b"}
Jun 02 04:42:54.540622 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.540589    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj" event={"ID":"36ec0192-f588-46d3-bb4d-ee4495c81eeb","Type":"ContainerStarted","Data":"7220d042b5b1e4ef1af959f26eafeba18512b4e01ea0d5af9522c835aa961f42"}
Jun 02 04:42:54.543166 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.543149    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 04:42:54.543493 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.543472    2577 generic.go:358] "Generic (PLEG): container finished" podID="88d63d7d-55eb-4340-99ae-9c49be3afdbe" containerID="fa395e9f7c300cdf0a0355bfd195f808b1fc5c010ad38d3c733edaffe7a479f7" exitCode=1
Jun 02 04:42:54.543568 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.543511    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc" event={"ID":"88d63d7d-55eb-4340-99ae-9c49be3afdbe","Type":"ContainerStarted","Data":"80787779d6130276fb54324bfd0e7b55ef8f7337d96068e0ca804b1a699a96b1"}
Jun 02 04:42:54.543568 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.543533    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc" event={"ID":"88d63d7d-55eb-4340-99ae-9c49be3afdbe","Type":"ContainerStarted","Data":"4c9cc35c2825840098d4c9c0a4f37a5d8ac9092bad741bd750a3885b83f9abf4"}
Jun 02 04:42:54.543568 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.543542    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc" event={"ID":"88d63d7d-55eb-4340-99ae-9c49be3afdbe","Type":"ContainerStarted","Data":"8ab87b0a865a39b15534b2550c8964a3452f0370a086d1b5c7c36ac49c5ead16"}
Jun 02 04:42:54.543568 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.543552    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc" event={"ID":"88d63d7d-55eb-4340-99ae-9c49be3afdbe","Type":"ContainerStarted","Data":"6921face86316d1451fb7096170682a5e188c4c09a0c8b8c10c50bd97b418f17"}
Jun 02 04:42:54.543568 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.543561    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc" event={"ID":"88d63d7d-55eb-4340-99ae-9c49be3afdbe","Type":"ContainerDied","Data":"fa395e9f7c300cdf0a0355bfd195f808b1fc5c010ad38d3c733edaffe7a479f7"}
Jun 02 04:42:54.543568 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.543574    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc" event={"ID":"88d63d7d-55eb-4340-99ae-9c49be3afdbe","Type":"ContainerStarted","Data":"f8d017882468b3531a7d49380a0700da01e344f3bad0bb0c91c4a961c39f81a7"}
Jun 02 04:42:54.544837 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.544802    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vfq72" event={"ID":"f99eac06-93b0-4b1b-aaa0-285d781e5412","Type":"ContainerStarted","Data":"2c7a29f150815e0049fefd5716bcc8bebaaae68d8a12c0a84b5b7682f3a9daee"}
Jun 02 04:42:54.544933 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.544848    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vfq72" event={"ID":"f99eac06-93b0-4b1b-aaa0-285d781e5412","Type":"ContainerStarted","Data":"0cff9c1ee54ab73d28b87d444327537ecd9f49d37a3f70f9e523db5823c31833"}
Jun 02 04:42:54.575018 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.574971    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-t4m5q" podStartSLOduration=3.7571425229999997 podStartE2EDuration="21.574956549s" podCreationTimestamp="2026-06-02 04:42:33 +0000 UTC" firstStartedPulling="2026-06-02 04:42:35.885218894 +0000 UTC m=+2.926149846" lastFinishedPulling="2026-06-02 04:42:53.703032921 +0000 UTC m=+20.743963872" observedRunningTime="2026-06-02 04:42:54.552981619 +0000 UTC m=+21.593912591" watchObservedRunningTime="2026-06-02 04:42:54.574956549 +0000 UTC m=+21.615887530"
Jun 02 04:42:54.608974 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.608937    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gk5rc" podStartSLOduration=3.879509947 podStartE2EDuration="21.608926572s" podCreationTimestamp="2026-06-02 04:42:33 +0000 UTC" firstStartedPulling="2026-06-02 04:42:35.88361147 +0000 UTC m=+2.924542419" lastFinishedPulling="2026-06-02 04:42:53.613028091 +0000 UTC m=+20.653959044" observedRunningTime="2026-06-02 04:42:54.588963943 +0000 UTC m=+21.629894916" watchObservedRunningTime="2026-06-02 04:42:54.608926572 +0000 UTC m=+21.649857544"
Jun 02 04:42:54.609370 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.609336    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qdfbx" podStartSLOduration=7.677381204 podStartE2EDuration="21.609324628s" podCreationTimestamp="2026-06-02 04:42:33 +0000 UTC" firstStartedPulling="2026-06-02 04:42:35.879260173 +0000 UTC m=+2.920191128" lastFinishedPulling="2026-06-02 04:42:49.811203583 +0000 UTC m=+16.852134552" observedRunningTime="2026-06-02 04:42:54.60866219 +0000 UTC m=+21.649593161" watchObservedRunningTime="2026-06-02 04:42:54.609324628 +0000 UTC m=+21.650255623"
Jun 02 04:42:54.626146 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.626107    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-vfq72" podStartSLOduration=13.626091679 podStartE2EDuration="13.626091679s" podCreationTimestamp="2026-06-02 04:42:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 04:42:54.625158806 +0000 UTC m=+21.666089789" watchObservedRunningTime="2026-06-02 04:42:54.626091679 +0000 UTC m=+21.667022652"
Jun 02 04:42:54.646323 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:54.646290    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-58cf2" podStartSLOduration=3.8597525 podStartE2EDuration="21.646280514s" podCreationTimestamp="2026-06-02 04:42:33 +0000 UTC" firstStartedPulling="2026-06-02 04:42:35.881899815 +0000 UTC m=+2.922830764" lastFinishedPulling="2026-06-02 04:42:53.668427822 +0000 UTC m=+20.709358778" observedRunningTime="2026-06-02 04:42:54.646046027 +0000 UTC m=+21.686977000" watchObservedRunningTime="2026-06-02 04:42:54.646280514 +0000 UTC m=+21.687211489"
Jun 02 04:42:55.006233 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:55.006195    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jun 02 04:42:55.182717 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:55.182642    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qdfbx"
Jun 02 04:42:55.402458 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:55.402317    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T04:42:55.006215331Z","UUID":"4185c0f9-12e5-418e-9d20-800ea7333597","Handler":null,"Name":"","Endpoint":""}
Jun 02 04:42:55.405800 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:55.405775    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jun 02 04:42:55.405940 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:55.405824    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jun 02 04:42:55.440203 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:55.440140    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:55.440331 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:55.440257    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5877c" podUID="f4b0125c-3e37-44fb-a810-b204058d5f45"
Jun 02 04:42:55.549053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:55.549007    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj" event={"ID":"36ec0192-f588-46d3-bb4d-ee4495c81eeb","Type":"ContainerStarted","Data":"4f0522bddf9231b70bf10f889a0a307898677ec0cbd5f7dc02e1d89a4fc0dda6"}
Jun 02 04:42:55.550487 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:55.550449    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v9tgv" event={"ID":"f19f54c7-a575-4537-aa41-524e5b5bf483","Type":"ContainerStarted","Data":"fcc37b8048104938deab095fd0074a2bcc9867301e93ce95dc97baba602ef4fa"}
Jun 02 04:42:55.570715 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:55.570617    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-v9tgv" podStartSLOduration=4.78815245 podStartE2EDuration="22.570603678s" podCreationTimestamp="2026-06-02 04:42:33 +0000 UTC" firstStartedPulling="2026-06-02 04:42:35.884672961 +0000 UTC m=+2.925603925" lastFinishedPulling="2026-06-02 04:42:53.667124184 +0000 UTC m=+20.708055153" observedRunningTime="2026-06-02 04:42:55.569970266 +0000 UTC m=+22.610901238" watchObservedRunningTime="2026-06-02 04:42:55.570603678 +0000 UTC m=+22.611534651"
Jun 02 04:42:56.439754 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:56.439723    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:56.439944 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:56.439859    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:42:56.554622 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:56.554576    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj" event={"ID":"36ec0192-f588-46d3-bb4d-ee4495c81eeb","Type":"ContainerStarted","Data":"71dad3851a8ac2b6544fe25a1e98d693197f1ed38d1ddf4e0786a21a4d0bf583"}
Jun 02 04:42:56.557828 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:56.557793    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 04:42:56.558567 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:56.558520    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc" event={"ID":"88d63d7d-55eb-4340-99ae-9c49be3afdbe","Type":"ContainerStarted","Data":"265d330778b2e90557b517588ae0fbea6916a24f48ea39fc8e8869daef37856e"}
Jun 02 04:42:56.589364 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:56.589318    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8ktnj" podStartSLOduration=3.698466131 podStartE2EDuration="23.589304494s" podCreationTimestamp="2026-06-02 04:42:33 +0000 UTC" firstStartedPulling="2026-06-02 04:42:35.875322636 +0000 UTC m=+2.916253587" lastFinishedPulling="2026-06-02 04:42:55.766160993 +0000 UTC m=+22.807091950" observedRunningTime="2026-06-02 04:42:56.587785045 +0000 UTC m=+23.628716016" watchObservedRunningTime="2026-06-02 04:42:56.589304494 +0000 UTC m=+23.630235469"
Jun 02 04:42:57.270300 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:57.270053    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qdfbx"
Jun 02 04:42:57.270750 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:57.270730    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qdfbx"
Jun 02 04:42:57.439904 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:57.439873    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:57.440053 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:57.439987    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5877c" podUID="f4b0125c-3e37-44fb-a810-b204058d5f45"
Jun 02 04:42:57.561471 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:57.561402    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qdfbx"
Jun 02 04:42:58.440334 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:58.440271    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:42:58.440457 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:58.440386    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:42:59.440224 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:59.440056    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:42:59.441051 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:42:59.440298    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5877c" podUID="f4b0125c-3e37-44fb-a810-b204058d5f45"
Jun 02 04:42:59.564868 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:59.564839    2577 generic.go:358] "Generic (PLEG): container finished" podID="dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd" containerID="c16f96d521cfce6650cf1607e68e474218e8062396b4df3898fcd664033d3636" exitCode=0
Jun 02 04:42:59.564999 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:59.564901    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29k6t" event={"ID":"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd","Type":"ContainerDied","Data":"c16f96d521cfce6650cf1607e68e474218e8062396b4df3898fcd664033d3636"}
Jun 02 04:42:59.567864 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:59.567849    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 04:42:59.568177 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:59.568143    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc" event={"ID":"88d63d7d-55eb-4340-99ae-9c49be3afdbe","Type":"ContainerStarted","Data":"19d8cf4815a1b826ae76521dd636b412ab023e14ac952df4d25ee0c6480b8751"}
Jun 02 04:42:59.568542 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:59.568523    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:59.568650 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:59.568552    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:42:59.568761 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:59.568745    2577 scope.go:117] "RemoveContainer" containerID="fa395e9f7c300cdf0a0355bfd195f808b1fc5c010ad38d3c733edaffe7a479f7"
Jun 02 04:42:59.584003 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:42:59.583984    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:43:00.439392 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:00.439361    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:43:00.439532 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:00.439507    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:43:00.571691 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:00.571665    2577 generic.go:358] "Generic (PLEG): container finished" podID="dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd" containerID="088fcfbb436ddd5c777764b01f271acf7e21897a8cc67c58066749e008cf4d4c" exitCode=0
Jun 02 04:43:00.572166 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:00.571745    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29k6t" event={"ID":"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd","Type":"ContainerDied","Data":"088fcfbb436ddd5c777764b01f271acf7e21897a8cc67c58066749e008cf4d4c"}
Jun 02 04:43:00.575403 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:00.575360    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 04:43:00.575740 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:00.575721    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc" event={"ID":"88d63d7d-55eb-4340-99ae-9c49be3afdbe","Type":"ContainerStarted","Data":"7d1260685b541bbce02cd65e1718bf6e6fff65a61f022612a19b8cf80c3c31cf"}
Jun 02 04:43:00.576110 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:00.576090    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:43:00.591488 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:00.591469    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:43:00.659521 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:00.659453    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc" podStartSLOduration=9.583258587 podStartE2EDuration="27.659439247s" podCreationTimestamp="2026-06-02 04:42:33 +0000 UTC" firstStartedPulling="2026-06-02 04:42:35.873878324 +0000 UTC m=+2.914809486" lastFinishedPulling="2026-06-02 04:42:53.950059196 +0000 UTC m=+20.990990146" observedRunningTime="2026-06-02 04:43:00.657942912 +0000 UTC m=+27.698873888" watchObservedRunningTime="2026-06-02 04:43:00.659439247 +0000 UTC m=+27.700370218"
Jun 02 04:43:00.834062 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:00.834033    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jnktq"]
Jun 02 04:43:00.834216 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:00.834145    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:43:00.834284 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:00.834225    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:43:00.836699 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:00.836673    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5877c"]
Jun 02 04:43:00.836846 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:00.836780    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:43:00.836914 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:00.836897    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5877c" podUID="f4b0125c-3e37-44fb-a810-b204058d5f45"
Jun 02 04:43:01.579153 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:01.578992    2577 generic.go:358] "Generic (PLEG): container finished" podID="dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd" containerID="d81c5ca8481d6603a816784aff8fa113cdd04dd1809f25865ac78e5ea73500e2" exitCode=0
Jun 02 04:43:01.579433 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:01.579071    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29k6t" event={"ID":"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd","Type":"ContainerDied","Data":"d81c5ca8481d6603a816784aff8fa113cdd04dd1809f25865ac78e5ea73500e2"}
Jun 02 04:43:02.440125 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:02.440095    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:43:02.440270 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:02.440097    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:43:02.440270 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:02.440194    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:43:02.440374 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:02.440285    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5877c" podUID="f4b0125c-3e37-44fb-a810-b204058d5f45"
Jun 02 04:43:04.440036 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:04.440008    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:43:04.440502 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:04.440002    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:43:04.440502 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:04.440133    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5877c" podUID="f4b0125c-3e37-44fb-a810-b204058d5f45"
Jun 02 04:43:04.440502 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:04.440214    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:43:06.270388 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.270355    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-23.ec2.internal" event="NodeReady"
Jun 02 04:43:06.270934 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.270501    2577 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
Jun 02 04:43:06.327854 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.327827    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zlpr5"]
Jun 02 04:43:06.359312 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.359271    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pzswt"]
Jun 02 04:43:06.359447 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.359384    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:06.366994 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.366347    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jun 02 04:43:06.366994 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.366509    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jun 02 04:43:06.367130 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.366993    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s8w52\""
Jun 02 04:43:06.373344 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.373324    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zlpr5"]
Jun 02 04:43:06.373432 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.373353    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pzswt"]
Jun 02 04:43:06.373432 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.373418    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:43:06.377555 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.377539    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-l292j\""
Jun 02 04:43:06.377651 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.377619    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jun 02 04:43:06.379632 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.379492    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jun 02 04:43:06.379632 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.379492    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jun 02 04:43:06.440088 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.440060    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:43:06.440228 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.440063    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:43:06.442987 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.442965    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jun 02 04:43:06.443108 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.443070    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2ckqz\""
Jun 02 04:43:06.443165 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.443110    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jun 02 04:43:06.443234 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.443221    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qgskw\""
Jun 02 04:43:06.443326 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.443302    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jun 02 04:43:06.484710 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.484652    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:06.484710 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.484704    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a1225d4-452b-41f9-9890-2e039b99c9d6-tmp-dir\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:06.484874 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.484754    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert\") pod \"ingress-canary-pzswt\" (UID: \"919222b9-64e3-4f4f-90c3-351f103d2a2f\") " pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:43:06.484874 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.484777    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5a1225d4-452b-41f9-9890-2e039b99c9d6-config-volume\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:06.484874 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.484802    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpf6f\" (UniqueName: \"kubernetes.io/projected/5a1225d4-452b-41f9-9890-2e039b99c9d6-kube-api-access-lpf6f\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:06.485003 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.484887    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5svkf\" (UniqueName: \"kubernetes.io/projected/919222b9-64e3-4f4f-90c3-351f103d2a2f-kube-api-access-5svkf\") pod \"ingress-canary-pzswt\" (UID: \"919222b9-64e3-4f4f-90c3-351f103d2a2f\") " pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:43:06.585475 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.585448    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5svkf\" (UniqueName: \"kubernetes.io/projected/919222b9-64e3-4f4f-90c3-351f103d2a2f-kube-api-access-5svkf\") pod \"ingress-canary-pzswt\" (UID: \"919222b9-64e3-4f4f-90c3-351f103d2a2f\") " pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:43:06.585623 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.585530    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:06.585623 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.585586    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a1225d4-452b-41f9-9890-2e039b99c9d6-tmp-dir\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:06.585623 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.585620    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert\") pod \"ingress-canary-pzswt\" (UID: \"919222b9-64e3-4f4f-90c3-351f103d2a2f\") " pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:43:06.585770 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.585643    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5a1225d4-452b-41f9-9890-2e039b99c9d6-config-volume\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:06.585770 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:06.585705    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 04:43:06.585770 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:06.585736    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 04:43:06.585934 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:06.585782    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls podName:5a1225d4-452b-41f9-9890-2e039b99c9d6 nodeName:}" failed. No retries permitted until 2026-06-02 04:43:07.085761993 +0000 UTC m=+34.126692949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls") pod "dns-default-zlpr5" (UID: "5a1225d4-452b-41f9-9890-2e039b99c9d6") : secret "dns-default-metrics-tls" not found
Jun 02 04:43:06.585934 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:06.585800    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert podName:919222b9-64e3-4f4f-90c3-351f103d2a2f nodeName:}" failed. No retries permitted until 2026-06-02 04:43:07.085791464 +0000 UTC m=+34.126722425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert") pod "ingress-canary-pzswt" (UID: "919222b9-64e3-4f4f-90c3-351f103d2a2f") : secret "canary-serving-cert" not found
Jun 02 04:43:06.585934 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.585834    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lpf6f\" (UniqueName: \"kubernetes.io/projected/5a1225d4-452b-41f9-9890-2e039b99c9d6-kube-api-access-lpf6f\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:06.586309 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.586283    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5a1225d4-452b-41f9-9890-2e039b99c9d6-config-volume\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:06.586435 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.586332    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a1225d4-452b-41f9-9890-2e039b99c9d6-tmp-dir\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:06.598220 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.598198    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5svkf\" (UniqueName: \"kubernetes.io/projected/919222b9-64e3-4f4f-90c3-351f103d2a2f-kube-api-access-5svkf\") pod \"ingress-canary-pzswt\" (UID: \"919222b9-64e3-4f4f-90c3-351f103d2a2f\") " pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:43:06.599522 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:06.599501    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpf6f\" (UniqueName: \"kubernetes.io/projected/5a1225d4-452b-41f9-9890-2e039b99c9d6-kube-api-access-lpf6f\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:07.089563 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:07.089533    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs\") pod \"network-metrics-daemon-jnktq\" (UID: \"7f26ffcc-136d-49e4-9072-626c6a31246e\") " pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:43:07.089716 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:07.089574    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert\") pod \"ingress-canary-pzswt\" (UID: \"919222b9-64e3-4f4f-90c3-351f103d2a2f\") " pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:43:07.089716 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:07.089610    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:07.089716 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:07.089678    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jun 02 04:43:07.092454 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:07.089932    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 04:43:07.092454 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:07.089974    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 04:43:07.092454 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:07.090018    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert podName:919222b9-64e3-4f4f-90c3-351f103d2a2f nodeName:}" failed. No retries permitted until 2026-06-02 04:43:08.089986391 +0000 UTC m=+35.130917345 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert") pod "ingress-canary-pzswt" (UID: "919222b9-64e3-4f4f-90c3-351f103d2a2f") : secret "canary-serving-cert" not found
Jun 02 04:43:07.092454 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:07.090290    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls podName:5a1225d4-452b-41f9-9890-2e039b99c9d6 nodeName:}" failed. No retries permitted until 2026-06-02 04:43:08.090270488 +0000 UTC m=+35.131201452 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls") pod "dns-default-zlpr5" (UID: "5a1225d4-452b-41f9-9890-2e039b99c9d6") : secret "dns-default-metrics-tls" not found
Jun 02 04:43:07.092454 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:07.090565    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs podName:7f26ffcc-136d-49e4-9072-626c6a31246e nodeName:}" failed. No retries permitted until 2026-06-02 04:43:39.090541928 +0000 UTC m=+66.131472889 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs") pod "network-metrics-daemon-jnktq" (UID: "7f26ffcc-136d-49e4-9072-626c6a31246e") : secret "metrics-daemon-secret" not found
Jun 02 04:43:07.190328 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:07.190304    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2pc67\" (UniqueName: \"kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67\") pod \"network-check-target-5877c\" (UID: \"f4b0125c-3e37-44fb-a810-b204058d5f45\") " pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:43:07.192739 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:07.192719    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pc67\" (UniqueName: \"kubernetes.io/projected/f4b0125c-3e37-44fb-a810-b204058d5f45-kube-api-access-2pc67\") pod \"network-check-target-5877c\" (UID: \"f4b0125c-3e37-44fb-a810-b204058d5f45\") " pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:43:07.351971 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:07.351920    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:43:07.523630 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:07.523379    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5877c"]
Jun 02 04:43:07.527857 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:43:07.527828    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4b0125c_3e37_44fb_a810_b204058d5f45.slice/crio-6e532495169d985ad0b53350fc206c6a82862e6e00cccbf23e9f278bdf698baf WatchSource:0}: Error finding container 6e532495169d985ad0b53350fc206c6a82862e6e00cccbf23e9f278bdf698baf: Status 404 returned error can't find the container with id 6e532495169d985ad0b53350fc206c6a82862e6e00cccbf23e9f278bdf698baf
Jun 02 04:43:07.592335 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:07.592299    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5877c" event={"ID":"f4b0125c-3e37-44fb-a810-b204058d5f45","Type":"ContainerStarted","Data":"6e532495169d985ad0b53350fc206c6a82862e6e00cccbf23e9f278bdf698baf"}
Jun 02 04:43:07.595090 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:07.595050    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29k6t" event={"ID":"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd","Type":"ContainerStarted","Data":"09df6ec2a3ee7f2fae56e2ce48330112a343ddffbdd0fc7ea6b4abd626e6ca45"}
Jun 02 04:43:08.097051 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:08.097014    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:08.097266 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:08.097173    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 04:43:08.097266 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:08.097184    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert\") pod \"ingress-canary-pzswt\" (UID: \"919222b9-64e3-4f4f-90c3-351f103d2a2f\") " pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:43:08.097266 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:08.097244    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls podName:5a1225d4-452b-41f9-9890-2e039b99c9d6 nodeName:}" failed. No retries permitted until 2026-06-02 04:43:10.097222926 +0000 UTC m=+37.138153881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls") pod "dns-default-zlpr5" (UID: "5a1225d4-452b-41f9-9890-2e039b99c9d6") : secret "dns-default-metrics-tls" not found
Jun 02 04:43:08.097435 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:08.097293    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 04:43:08.097435 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:08.097349    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert podName:919222b9-64e3-4f4f-90c3-351f103d2a2f nodeName:}" failed. No retries permitted until 2026-06-02 04:43:10.097333321 +0000 UTC m=+37.138264276 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert") pod "ingress-canary-pzswt" (UID: "919222b9-64e3-4f4f-90c3-351f103d2a2f") : secret "canary-serving-cert" not found
Jun 02 04:43:08.599751 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:08.599720    2577 generic.go:358] "Generic (PLEG): container finished" podID="dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd" containerID="09df6ec2a3ee7f2fae56e2ce48330112a343ddffbdd0fc7ea6b4abd626e6ca45" exitCode=0
Jun 02 04:43:08.600209 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:08.599789    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29k6t" event={"ID":"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd","Type":"ContainerDied","Data":"09df6ec2a3ee7f2fae56e2ce48330112a343ddffbdd0fc7ea6b4abd626e6ca45"}
Jun 02 04:43:09.605835 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:09.605786    2577 generic.go:358] "Generic (PLEG): container finished" podID="dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd" containerID="d748524bfac6d7c233f157ea671e07d146eb3535a66a070d67ef95d575f4a822" exitCode=0
Jun 02 04:43:09.606262 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:09.605847    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29k6t" event={"ID":"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd","Type":"ContainerDied","Data":"d748524bfac6d7c233f157ea671e07d146eb3535a66a070d67ef95d575f4a822"}
Jun 02 04:43:10.113713 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:10.113679    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert\") pod \"ingress-canary-pzswt\" (UID: \"919222b9-64e3-4f4f-90c3-351f103d2a2f\") " pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:43:10.113927 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:10.113747    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:10.113927 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:10.113862    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 04:43:10.113927 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:10.113926    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls podName:5a1225d4-452b-41f9-9890-2e039b99c9d6 nodeName:}" failed. No retries permitted until 2026-06-02 04:43:14.113909884 +0000 UTC m=+41.154840859 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls") pod "dns-default-zlpr5" (UID: "5a1225d4-452b-41f9-9890-2e039b99c9d6") : secret "dns-default-metrics-tls" not found
Jun 02 04:43:10.114098 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:10.113862    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 04:43:10.114098 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:10.113985    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert podName:919222b9-64e3-4f4f-90c3-351f103d2a2f nodeName:}" failed. No retries permitted until 2026-06-02 04:43:14.113972705 +0000 UTC m=+41.154903659 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert") pod "ingress-canary-pzswt" (UID: "919222b9-64e3-4f4f-90c3-351f103d2a2f") : secret "canary-serving-cert" not found
Jun 02 04:43:10.608798 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:10.608638    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5877c" event={"ID":"f4b0125c-3e37-44fb-a810-b204058d5f45","Type":"ContainerStarted","Data":"1ddfb6fe59d6c7c076c4daa9719a191d343d5705d7b1cb5258cfcca0989a60ec"}
Jun 02 04:43:10.609207 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:10.608805    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:43:10.611413 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:10.611390    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29k6t" event={"ID":"dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd","Type":"ContainerStarted","Data":"91858dc254257d0be9e6e204db180deaf6094bae5d745c07d86abe2735cd879f"}
Jun 02 04:43:10.656402 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:10.656352    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-29k6t" podStartSLOduration=6.199338479 podStartE2EDuration="37.656335716s" podCreationTimestamp="2026-06-02 04:42:33 +0000 UTC" firstStartedPulling="2026-06-02 04:42:35.880972443 +0000 UTC m=+2.921903398" lastFinishedPulling="2026-06-02 04:43:07.337969682 +0000 UTC m=+34.378900635" observedRunningTime="2026-06-02 04:43:10.655535378 +0000 UTC m=+37.696466350" watchObservedRunningTime="2026-06-02 04:43:10.656335716 +0000 UTC m=+37.697266689"
Jun 02 04:43:10.656545 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:10.656508    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5877c" podStartSLOduration=34.766939157 podStartE2EDuration="37.656500649s" podCreationTimestamp="2026-06-02 04:42:33 +0000 UTC" firstStartedPulling="2026-06-02 04:43:07.529705271 +0000 UTC m=+34.570636225" lastFinishedPulling="2026-06-02 04:43:10.419266767 +0000 UTC m=+37.460197717" observedRunningTime="2026-06-02 04:43:10.633219152 +0000 UTC m=+37.674150123" watchObservedRunningTime="2026-06-02 04:43:10.656500649 +0000 UTC m=+37.697431621"
Jun 02 04:43:14.140130 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:14.140098    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:14.140547 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:14.140145    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert\") pod \"ingress-canary-pzswt\" (UID: \"919222b9-64e3-4f4f-90c3-351f103d2a2f\") " pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:43:14.140547 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:14.140227    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 04:43:14.140547 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:14.140229    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 04:43:14.140547 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:14.140274    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls podName:5a1225d4-452b-41f9-9890-2e039b99c9d6 nodeName:}" failed. No retries permitted until 2026-06-02 04:43:22.140259863 +0000 UTC m=+49.181190830 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls") pod "dns-default-zlpr5" (UID: "5a1225d4-452b-41f9-9890-2e039b99c9d6") : secret "dns-default-metrics-tls" not found
Jun 02 04:43:14.140547 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:14.140287    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert podName:919222b9-64e3-4f4f-90c3-351f103d2a2f nodeName:}" failed. No retries permitted until 2026-06-02 04:43:22.140280406 +0000 UTC m=+49.181211355 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert") pod "ingress-canary-pzswt" (UID: "919222b9-64e3-4f4f-90c3-351f103d2a2f") : secret "canary-serving-cert" not found
Jun 02 04:43:22.167580 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.167548    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d458f69fb-jwbd5"]
Jun 02 04:43:22.178786 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.178758    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d458f69fb-jwbd5"
Jun 02 04:43:22.180096 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.180074    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"]
Jun 02 04:43:22.184332 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.183252    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jun 02 04:43:22.184332 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.183401    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jun 02 04:43:22.184778 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.184688    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-fcb9h\""
Jun 02 04:43:22.184892 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.184775    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jun 02 04:43:22.184892 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.184882    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jun 02 04:43:22.188917 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.188897    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert\") pod \"ingress-canary-pzswt\" (UID: \"919222b9-64e3-4f4f-90c3-351f103d2a2f\") " pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:43:22.189019 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.188952    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:22.189079 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:22.189022    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 04:43:22.189079 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:22.189067    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 04:43:22.189079 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:22.189078    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert podName:919222b9-64e3-4f4f-90c3-351f103d2a2f nodeName:}" failed. No retries permitted until 2026-06-02 04:43:38.189065376 +0000 UTC m=+65.229996326 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert") pod "ingress-canary-pzswt" (UID: "919222b9-64e3-4f4f-90c3-351f103d2a2f") : secret "canary-serving-cert" not found
Jun 02 04:43:22.189272 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:22.189146    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls podName:5a1225d4-452b-41f9-9890-2e039b99c9d6 nodeName:}" failed. No retries permitted until 2026-06-02 04:43:38.189109212 +0000 UTC m=+65.230040162 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls") pod "dns-default-zlpr5" (UID: "5a1225d4-452b-41f9-9890-2e039b99c9d6") : secret "dns-default-metrics-tls" not found
Jun 02 04:43:22.197473 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.197456    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d458f69fb-jwbd5"]
Jun 02 04:43:22.197473 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.197476    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"]
Jun 02 04:43:22.197596 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.197549    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.200418 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.200399    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jun 02 04:43:22.200662 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.200645    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jun 02 04:43:22.200728 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.200677    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jun 02 04:43:22.200728 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.200708    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jun 02 04:43:22.290042 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.290021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hg9b\" (UniqueName: \"kubernetes.io/projected/91a06157-a8f2-4bc0-b06d-a6d288988fe2-kube-api-access-7hg9b\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.290125 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.290056    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/91a06157-a8f2-4bc0-b06d-a6d288988fe2-ca\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.290125 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.290078    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/91a06157-a8f2-4bc0-b06d-a6d288988fe2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.290196 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.290162    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/91a06157-a8f2-4bc0-b06d-a6d288988fe2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.290236 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.290216    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0477b761-d77b-460d-b073-05e27aebdffe-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5d458f69fb-jwbd5\" (UID: \"0477b761-d77b-460d-b073-05e27aebdffe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d458f69fb-jwbd5"
Jun 02 04:43:22.290269 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.290254    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67wqh\" (UniqueName: \"kubernetes.io/projected/0477b761-d77b-460d-b073-05e27aebdffe-kube-api-access-67wqh\") pod \"managed-serviceaccount-addon-agent-5d458f69fb-jwbd5\" (UID: \"0477b761-d77b-460d-b073-05e27aebdffe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d458f69fb-jwbd5"
Jun 02 04:43:22.290304 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.290270    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/91a06157-a8f2-4bc0-b06d-a6d288988fe2-hub\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.290304 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.290287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/91a06157-a8f2-4bc0-b06d-a6d288988fe2-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.391118 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.391099    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7hg9b\" (UniqueName: \"kubernetes.io/projected/91a06157-a8f2-4bc0-b06d-a6d288988fe2-kube-api-access-7hg9b\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.391181 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.391137    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/91a06157-a8f2-4bc0-b06d-a6d288988fe2-ca\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.391181 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.391159    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/91a06157-a8f2-4bc0-b06d-a6d288988fe2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.391249 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.391204    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/91a06157-a8f2-4bc0-b06d-a6d288988fe2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.391249 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.391238    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0477b761-d77b-460d-b073-05e27aebdffe-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5d458f69fb-jwbd5\" (UID: \"0477b761-d77b-460d-b073-05e27aebdffe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d458f69fb-jwbd5"
Jun 02 04:43:22.391332 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.391267    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-67wqh\" (UniqueName: \"kubernetes.io/projected/0477b761-d77b-460d-b073-05e27aebdffe-kube-api-access-67wqh\") pod \"managed-serviceaccount-addon-agent-5d458f69fb-jwbd5\" (UID: \"0477b761-d77b-460d-b073-05e27aebdffe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d458f69fb-jwbd5"
Jun 02 04:43:22.391332 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.391307    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/91a06157-a8f2-4bc0-b06d-a6d288988fe2-hub\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.391426 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.391335    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/91a06157-a8f2-4bc0-b06d-a6d288988fe2-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.392006 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.391984    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/91a06157-a8f2-4bc0-b06d-a6d288988fe2-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.395321 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.395300    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/91a06157-a8f2-4bc0-b06d-a6d288988fe2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.395419 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.395318    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/91a06157-a8f2-4bc0-b06d-a6d288988fe2-ca\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.395419 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.395391    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0477b761-d77b-460d-b073-05e27aebdffe-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5d458f69fb-jwbd5\" (UID: \"0477b761-d77b-460d-b073-05e27aebdffe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d458f69fb-jwbd5"
Jun 02 04:43:22.395517 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.395426    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/91a06157-a8f2-4bc0-b06d-a6d288988fe2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.395517 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.395433    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/91a06157-a8f2-4bc0-b06d-a6d288988fe2-hub\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.399648 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.399626    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67wqh\" (UniqueName: \"kubernetes.io/projected/0477b761-d77b-460d-b073-05e27aebdffe-kube-api-access-67wqh\") pod \"managed-serviceaccount-addon-agent-5d458f69fb-jwbd5\" (UID: \"0477b761-d77b-460d-b073-05e27aebdffe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d458f69fb-jwbd5"
Jun 02 04:43:22.399953 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.399936    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hg9b\" (UniqueName: \"kubernetes.io/projected/91a06157-a8f2-4bc0-b06d-a6d288988fe2-kube-api-access-7hg9b\") pod \"cluster-proxy-proxy-agent-5f75b45677-79546\" (UID: \"91a06157-a8f2-4bc0-b06d-a6d288988fe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.498775 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.498756    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d458f69fb-jwbd5"
Jun 02 04:43:22.506411 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.506369    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:43:22.642989 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.642961    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d458f69fb-jwbd5"]
Jun 02 04:43:22.647882 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:43:22.647860    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0477b761_d77b_460d_b073_05e27aebdffe.slice/crio-1d884320cff676db8264b4bc68bd1687ba251c63ea271b82e95ee28f2f1bafca WatchSource:0}: Error finding container 1d884320cff676db8264b4bc68bd1687ba251c63ea271b82e95ee28f2f1bafca: Status 404 returned error can't find the container with id 1d884320cff676db8264b4bc68bd1687ba251c63ea271b82e95ee28f2f1bafca
Jun 02 04:43:22.664984 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:22.664961    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"]
Jun 02 04:43:22.667986 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:43:22.667962    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91a06157_a8f2_4bc0_b06d_a6d288988fe2.slice/crio-8cadd2be1e28e90768dbde195fb93dacd8a6eb50476792dc824d10b132a79604 WatchSource:0}: Error finding container 8cadd2be1e28e90768dbde195fb93dacd8a6eb50476792dc824d10b132a79604: Status 404 returned error can't find the container with id 8cadd2be1e28e90768dbde195fb93dacd8a6eb50476792dc824d10b132a79604
Jun 02 04:43:23.646775 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:23.646691    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546" event={"ID":"91a06157-a8f2-4bc0-b06d-a6d288988fe2","Type":"ContainerStarted","Data":"8cadd2be1e28e90768dbde195fb93dacd8a6eb50476792dc824d10b132a79604"}
Jun 02 04:43:23.648437 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:23.648372    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d458f69fb-jwbd5" event={"ID":"0477b761-d77b-460d-b073-05e27aebdffe","Type":"ContainerStarted","Data":"1d884320cff676db8264b4bc68bd1687ba251c63ea271b82e95ee28f2f1bafca"}
Jun 02 04:43:27.659004 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:27.658947    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546" event={"ID":"91a06157-a8f2-4bc0-b06d-a6d288988fe2","Type":"ContainerStarted","Data":"7df468b4ed46949b10bf55ab0cace0604e6350053a8aba104630f316e32ff956"}
Jun 02 04:43:27.660583 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:27.660512    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d458f69fb-jwbd5" event={"ID":"0477b761-d77b-460d-b073-05e27aebdffe","Type":"ContainerStarted","Data":"bbb0bd044106b54d80e3a1018e84f2c0731a285dcd8b8d4651e06e45b4a1fd20"}
Jun 02 04:43:30.668300 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:30.668267    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546" event={"ID":"91a06157-a8f2-4bc0-b06d-a6d288988fe2","Type":"ContainerStarted","Data":"b6c37e8f03079ae9dfa993935b3901d0bfdabbb42cedd2cd946e5778eeeae8b2"}
Jun 02 04:43:30.668300 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:30.668303    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546" event={"ID":"91a06157-a8f2-4bc0-b06d-a6d288988fe2","Type":"ContainerStarted","Data":"ca468e742f6cb4203aaedc7141c2decf96c4bf934f70fbda41e17ec7e59f9c61"}
Jun 02 04:43:30.689417 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:30.689369    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d458f69fb-jwbd5" podStartSLOduration=4.701978149 podStartE2EDuration="8.689356262s" podCreationTimestamp="2026-06-02 04:43:22 +0000 UTC" firstStartedPulling="2026-06-02 04:43:22.649581823 +0000 UTC m=+49.690512777" lastFinishedPulling="2026-06-02 04:43:26.63695993 +0000 UTC m=+53.677890890" observedRunningTime="2026-06-02 04:43:27.677427899 +0000 UTC m=+54.718358871" watchObservedRunningTime="2026-06-02 04:43:30.689356262 +0000 UTC m=+57.730287233"
Jun 02 04:43:30.689579 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:30.689446    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546" podStartSLOduration=1.646329026 podStartE2EDuration="8.689440544s" podCreationTimestamp="2026-06-02 04:43:22 +0000 UTC" firstStartedPulling="2026-06-02 04:43:22.669570234 +0000 UTC m=+49.710501188" lastFinishedPulling="2026-06-02 04:43:29.712681746 +0000 UTC m=+56.753612706" observedRunningTime="2026-06-02 04:43:30.687771792 +0000 UTC m=+57.728702760" watchObservedRunningTime="2026-06-02 04:43:30.689440544 +0000 UTC m=+57.730371516"
Jun 02 04:43:32.591506 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:32.591480    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hgcxc"
Jun 02 04:43:38.201623 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:38.201588    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:43:38.202079 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:38.201639    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert\") pod \"ingress-canary-pzswt\" (UID: \"919222b9-64e3-4f4f-90c3-351f103d2a2f\") " pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:43:38.202079 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:38.201720    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 04:43:38.202079 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:38.201720    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 04:43:38.202079 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:38.201777    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert podName:919222b9-64e3-4f4f-90c3-351f103d2a2f nodeName:}" failed. No retries permitted until 2026-06-02 04:44:10.201763491 +0000 UTC m=+97.242694441 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert") pod "ingress-canary-pzswt" (UID: "919222b9-64e3-4f4f-90c3-351f103d2a2f") : secret "canary-serving-cert" not found
Jun 02 04:43:38.202079 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:38.201789    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls podName:5a1225d4-452b-41f9-9890-2e039b99c9d6 nodeName:}" failed. No retries permitted until 2026-06-02 04:44:10.201783396 +0000 UTC m=+97.242714346 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls") pod "dns-default-zlpr5" (UID: "5a1225d4-452b-41f9-9890-2e039b99c9d6") : secret "dns-default-metrics-tls" not found
Jun 02 04:43:39.108077 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:39.108043    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs\") pod \"network-metrics-daemon-jnktq\" (UID: \"7f26ffcc-136d-49e4-9072-626c6a31246e\") " pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:43:39.108230 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:39.108147    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jun 02 04:43:39.108230 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:43:39.108206    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs podName:7f26ffcc-136d-49e4-9072-626c6a31246e nodeName:}" failed. No retries permitted until 2026-06-02 04:44:43.108191833 +0000 UTC m=+130.149122788 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs") pod "network-metrics-daemon-jnktq" (UID: "7f26ffcc-136d-49e4-9072-626c6a31246e") : secret "metrics-daemon-secret" not found
Jun 02 04:43:41.614882 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:43:41.614853    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5877c"
Jun 02 04:44:10.214270 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:44:10.214164    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:44:10.214270 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:44:10.214217    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert\") pod \"ingress-canary-pzswt\" (UID: \"919222b9-64e3-4f4f-90c3-351f103d2a2f\") " pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:44:10.214736 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:44:10.214301    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
Jun 02 04:44:10.214736 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:44:10.214367    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert podName:919222b9-64e3-4f4f-90c3-351f103d2a2f nodeName:}" failed. No retries permitted until 2026-06-02 04:45:14.214350297 +0000 UTC m=+161.255281270 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert") pod "ingress-canary-pzswt" (UID: "919222b9-64e3-4f4f-90c3-351f103d2a2f") : secret "canary-serving-cert" not found
Jun 02 04:44:10.214736 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:44:10.214301    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jun 02 04:44:10.214736 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:44:10.214446    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls podName:5a1225d4-452b-41f9-9890-2e039b99c9d6 nodeName:}" failed. No retries permitted until 2026-06-02 04:45:14.214430336 +0000 UTC m=+161.255361300 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls") pod "dns-default-zlpr5" (UID: "5a1225d4-452b-41f9-9890-2e039b99c9d6") : secret "dns-default-metrics-tls" not found
Jun 02 04:44:43.126116 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:44:43.126079    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs\") pod \"network-metrics-daemon-jnktq\" (UID: \"7f26ffcc-136d-49e4-9072-626c6a31246e\") " pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:44:43.126559 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:44:43.126202    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jun 02 04:44:43.126559 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:44:43.126269    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs podName:7f26ffcc-136d-49e4-9072-626c6a31246e nodeName:}" failed. No retries permitted until 2026-06-02 04:46:45.126254599 +0000 UTC m=+252.167185554 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs") pod "network-metrics-daemon-jnktq" (UID: "7f26ffcc-136d-49e4-9072-626c6a31246e") : secret "metrics-daemon-secret" not found
Jun 02 04:44:59.843040 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:44:59.843014    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vfq72_f99eac06-93b0-4b1b-aaa0-285d781e5412/dns-node-resolver/0.log"
Jun 02 04:45:00.442266 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:00.442241    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gk5rc_a77859ea-6f29-411d-b3d1-abc2e33e3745/node-ca/0.log"
Jun 02 04:45:09.370275 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:45:09.370238    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-zlpr5" podUID="5a1225d4-452b-41f9-9890-2e039b99c9d6"
Jun 02 04:45:09.383391 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:45:09.383370    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-pzswt" podUID="919222b9-64e3-4f4f-90c3-351f103d2a2f"
Jun 02 04:45:09.457550 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:45:09.457523    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-jnktq" podUID="7f26ffcc-136d-49e4-9072-626c6a31246e"
Jun 02 04:45:09.846539 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:09.846513    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:45:09.846667 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:09.846517    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:45:14.233271 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:14.233238    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert\") pod \"ingress-canary-pzswt\" (UID: \"919222b9-64e3-4f4f-90c3-351f103d2a2f\") " pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:45:14.233656 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:14.233282    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:45:14.235587 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:14.235560    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/919222b9-64e3-4f4f-90c3-351f103d2a2f-cert\") pod \"ingress-canary-pzswt\" (UID: \"919222b9-64e3-4f4f-90c3-351f103d2a2f\") " pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:45:14.235711 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:14.235594    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a1225d4-452b-41f9-9890-2e039b99c9d6-metrics-tls\") pod \"dns-default-zlpr5\" (UID: \"5a1225d4-452b-41f9-9890-2e039b99c9d6\") " pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:45:14.351454 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:14.351432    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s8w52\""
Jun 02 04:45:14.351584 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:14.351432    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-l292j\""
Jun 02 04:45:14.357995 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:14.357981    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pzswt"
Jun 02 04:45:14.358100 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:14.358079    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:45:14.502892 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:14.502865    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pzswt"]
Jun 02 04:45:14.505575 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:45:14.505548    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod919222b9_64e3_4f4f_90c3_351f103d2a2f.slice/crio-f3de43fec6395665578d580ebc5ff933597cc370541810ba1adb019e08355790 WatchSource:0}: Error finding container f3de43fec6395665578d580ebc5ff933597cc370541810ba1adb019e08355790: Status 404 returned error can't find the container with id f3de43fec6395665578d580ebc5ff933597cc370541810ba1adb019e08355790
Jun 02 04:45:14.518005 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:14.517986    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zlpr5"]
Jun 02 04:45:14.519688 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:45:14.519666    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a1225d4_452b_41f9_9890_2e039b99c9d6.slice/crio-7080c0180d3688ba06dbbbd741818853d46ca35218f82214f9ee7aea5f292267 WatchSource:0}: Error finding container 7080c0180d3688ba06dbbbd741818853d46ca35218f82214f9ee7aea5f292267: Status 404 returned error can't find the container with id 7080c0180d3688ba06dbbbd741818853d46ca35218f82214f9ee7aea5f292267
Jun 02 04:45:14.855982 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:14.855915    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pzswt" event={"ID":"919222b9-64e3-4f4f-90c3-351f103d2a2f","Type":"ContainerStarted","Data":"f3de43fec6395665578d580ebc5ff933597cc370541810ba1adb019e08355790"}
Jun 02 04:45:14.856789 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:14.856768    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zlpr5" event={"ID":"5a1225d4-452b-41f9-9890-2e039b99c9d6","Type":"ContainerStarted","Data":"7080c0180d3688ba06dbbbd741818853d46ca35218f82214f9ee7aea5f292267"}
Jun 02 04:45:16.862849 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:16.862804    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zlpr5" event={"ID":"5a1225d4-452b-41f9-9890-2e039b99c9d6","Type":"ContainerStarted","Data":"8da0c8e5d3d745b4dd0c571d073dccc294f6bb9ff2f93f7ffbff2e13c3fe03e1"}
Jun 02 04:45:16.862849 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:16.862848    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zlpr5" event={"ID":"5a1225d4-452b-41f9-9890-2e039b99c9d6","Type":"ContainerStarted","Data":"be8d776fb26cf7b7a2a4fef3399d5528226b261706227a815dc1ddcd8753c1e3"}
Jun 02 04:45:16.863270 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:16.862911    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:45:16.864034 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:16.864013    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pzswt" event={"ID":"919222b9-64e3-4f4f-90c3-351f103d2a2f","Type":"ContainerStarted","Data":"10de645099cca1e97f77f003040d6ebd6e56065461458724485302e726d08a53"}
Jun 02 04:45:16.898404 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:16.898368    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pzswt" podStartSLOduration=129.137984205 podStartE2EDuration="2m10.898356945s" podCreationTimestamp="2026-06-02 04:43:06 +0000 UTC" firstStartedPulling="2026-06-02 04:45:14.507262207 +0000 UTC m=+161.548193169" lastFinishedPulling="2026-06-02 04:45:16.267634957 +0000 UTC m=+163.308565909" observedRunningTime="2026-06-02 04:45:16.898345677 +0000 UTC m=+163.939276652" watchObservedRunningTime="2026-06-02 04:45:16.898356945 +0000 UTC m=+163.939287913"
Jun 02 04:45:16.898874 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:16.898849    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-zlpr5" podStartSLOduration=129.155934007 podStartE2EDuration="2m10.898840309s" podCreationTimestamp="2026-06-02 04:43:06 +0000 UTC" firstStartedPulling="2026-06-02 04:45:14.521326523 +0000 UTC m=+161.562257472" lastFinishedPulling="2026-06-02 04:45:16.264232812 +0000 UTC m=+163.305163774" observedRunningTime="2026-06-02 04:45:16.881957786 +0000 UTC m=+163.922888757" watchObservedRunningTime="2026-06-02 04:45:16.898840309 +0000 UTC m=+163.939771280"
Jun 02 04:45:20.439848 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:20.439777    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:45:24.725677 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.725603    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rbpq7"]
Jun 02 04:45:24.728614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.728595    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:24.732662 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.732629    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jun 02 04:45:24.732662 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.732650    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jun 02 04:45:24.732850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.732646    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bxtwx\""
Jun 02 04:45:24.732850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.732629    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jun 02 04:45:24.733606 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.733591    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jun 02 04:45:24.742410 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.742387    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rbpq7"]
Jun 02 04:45:24.798861 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.798843    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkmnr\" (UniqueName: \"kubernetes.io/projected/eb5b0c03-aba6-4531-a125-377ffc9f68e4-kube-api-access-qkmnr\") pod \"insights-runtime-extractor-rbpq7\" (UID: \"eb5b0c03-aba6-4531-a125-377ffc9f68e4\") " pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:24.798973 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.798881    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb5b0c03-aba6-4531-a125-377ffc9f68e4-crio-socket\") pod \"insights-runtime-extractor-rbpq7\" (UID: \"eb5b0c03-aba6-4531-a125-377ffc9f68e4\") " pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:24.798973 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.798903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb5b0c03-aba6-4531-a125-377ffc9f68e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rbpq7\" (UID: \"eb5b0c03-aba6-4531-a125-377ffc9f68e4\") " pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:24.799507 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.799489    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb5b0c03-aba6-4531-a125-377ffc9f68e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rbpq7\" (UID: \"eb5b0c03-aba6-4531-a125-377ffc9f68e4\") " pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:24.799547 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.799517    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb5b0c03-aba6-4531-a125-377ffc9f68e4-data-volume\") pod \"insights-runtime-extractor-rbpq7\" (UID: \"eb5b0c03-aba6-4531-a125-377ffc9f68e4\") " pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:24.840415 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.840392    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7494f5fddc-msbxk"]
Jun 02 04:45:24.844224 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.844210    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:24.847332 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.847315    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jun 02 04:45:24.847559 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.847540    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4vpfz\""
Jun 02 04:45:24.847559 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.847553    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jun 02 04:45:24.849434 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.849419    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jun 02 04:45:24.869902 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.869884    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jun 02 04:45:24.873600 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.873580    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7494f5fddc-msbxk"]
Jun 02 04:45:24.899777 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.899754    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qkmnr\" (UniqueName: \"kubernetes.io/projected/eb5b0c03-aba6-4531-a125-377ffc9f68e4-kube-api-access-qkmnr\") pod \"insights-runtime-extractor-rbpq7\" (UID: \"eb5b0c03-aba6-4531-a125-377ffc9f68e4\") " pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:24.899881 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.899786    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17dfd930-9652-4845-a83a-9f40ea622fcd-image-registry-private-configuration\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:24.899881 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.899805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17dfd930-9652-4845-a83a-9f40ea622fcd-registry-tls\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:24.900012 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.899895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17dfd930-9652-4845-a83a-9f40ea622fcd-ca-trust-extracted\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:24.900012 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.899945    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb5b0c03-aba6-4531-a125-377ffc9f68e4-crio-socket\") pod \"insights-runtime-extractor-rbpq7\" (UID: \"eb5b0c03-aba6-4531-a125-377ffc9f68e4\") " pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:24.900012 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.899973    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17dfd930-9652-4845-a83a-9f40ea622fcd-bound-sa-token\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:24.900012 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.899985    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb5b0c03-aba6-4531-a125-377ffc9f68e4-crio-socket\") pod \"insights-runtime-extractor-rbpq7\" (UID: \"eb5b0c03-aba6-4531-a125-377ffc9f68e4\") " pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:24.900210 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.900011    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb5b0c03-aba6-4531-a125-377ffc9f68e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rbpq7\" (UID: \"eb5b0c03-aba6-4531-a125-377ffc9f68e4\") " pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:24.900210 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.900061    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb5b0c03-aba6-4531-a125-377ffc9f68e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rbpq7\" (UID: \"eb5b0c03-aba6-4531-a125-377ffc9f68e4\") " pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:24.900210 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.900093    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17dfd930-9652-4845-a83a-9f40ea622fcd-installation-pull-secrets\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:24.900210 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.900129    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb5b0c03-aba6-4531-a125-377ffc9f68e4-data-volume\") pod \"insights-runtime-extractor-rbpq7\" (UID: \"eb5b0c03-aba6-4531-a125-377ffc9f68e4\") " pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:24.900210 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.900163    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17dfd930-9652-4845-a83a-9f40ea622fcd-trusted-ca\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:24.900397 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.900215    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17dfd930-9652-4845-a83a-9f40ea622fcd-registry-certificates\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:24.900397 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.900241    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lrzn\" (UniqueName: \"kubernetes.io/projected/17dfd930-9652-4845-a83a-9f40ea622fcd-kube-api-access-7lrzn\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:24.900397 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.900367    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb5b0c03-aba6-4531-a125-377ffc9f68e4-data-volume\") pod \"insights-runtime-extractor-rbpq7\" (UID: \"eb5b0c03-aba6-4531-a125-377ffc9f68e4\") " pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:24.900574 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.900556    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb5b0c03-aba6-4531-a125-377ffc9f68e4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rbpq7\" (UID: \"eb5b0c03-aba6-4531-a125-377ffc9f68e4\") " pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:24.902370 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.902354    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb5b0c03-aba6-4531-a125-377ffc9f68e4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rbpq7\" (UID: \"eb5b0c03-aba6-4531-a125-377ffc9f68e4\") " pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:24.910324 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:24.910302    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkmnr\" (UniqueName: \"kubernetes.io/projected/eb5b0c03-aba6-4531-a125-377ffc9f68e4-kube-api-access-qkmnr\") pod \"insights-runtime-extractor-rbpq7\" (UID: \"eb5b0c03-aba6-4531-a125-377ffc9f68e4\") " pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:25.001308 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.001289    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17dfd930-9652-4845-a83a-9f40ea622fcd-registry-certificates\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.001376 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.001315    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7lrzn\" (UniqueName: \"kubernetes.io/projected/17dfd930-9652-4845-a83a-9f40ea622fcd-kube-api-access-7lrzn\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.001376 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.001338    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17dfd930-9652-4845-a83a-9f40ea622fcd-image-registry-private-configuration\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.001376 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.001355    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17dfd930-9652-4845-a83a-9f40ea622fcd-registry-tls\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.001514 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.001381    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17dfd930-9652-4845-a83a-9f40ea622fcd-ca-trust-extracted\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.001514 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.001409    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17dfd930-9652-4845-a83a-9f40ea622fcd-bound-sa-token\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.001514 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.001444    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17dfd930-9652-4845-a83a-9f40ea622fcd-installation-pull-secrets\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.001514 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.001470    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17dfd930-9652-4845-a83a-9f40ea622fcd-trusted-ca\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.001893 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.001867    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17dfd930-9652-4845-a83a-9f40ea622fcd-ca-trust-extracted\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.002214 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.002180    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17dfd930-9652-4845-a83a-9f40ea622fcd-registry-certificates\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.002433 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.002415    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17dfd930-9652-4845-a83a-9f40ea622fcd-trusted-ca\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.003577 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.003552    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17dfd930-9652-4845-a83a-9f40ea622fcd-image-registry-private-configuration\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.003832 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.003798    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17dfd930-9652-4845-a83a-9f40ea622fcd-registry-tls\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.003908 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.003833    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17dfd930-9652-4845-a83a-9f40ea622fcd-installation-pull-secrets\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.011046 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.011020    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lrzn\" (UniqueName: \"kubernetes.io/projected/17dfd930-9652-4845-a83a-9f40ea622fcd-kube-api-access-7lrzn\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.011861 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.011846    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17dfd930-9652-4845-a83a-9f40ea622fcd-bound-sa-token\") pod \"image-registry-7494f5fddc-msbxk\" (UID: \"17dfd930-9652-4845-a83a-9f40ea622fcd\") " pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.037263 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.037240    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rbpq7"
Jun 02 04:45:25.152793 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.152765    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.160420 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.160388    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rbpq7"]
Jun 02 04:45:25.163766 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:45:25.163740    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb5b0c03_aba6_4531_a125_377ffc9f68e4.slice/crio-0fb0e5f3323f64f32d7d6ccd9b2d4069b8eb0ff9e50c05945a2dc687300f362e WatchSource:0}: Error finding container 0fb0e5f3323f64f32d7d6ccd9b2d4069b8eb0ff9e50c05945a2dc687300f362e: Status 404 returned error can't find the container with id 0fb0e5f3323f64f32d7d6ccd9b2d4069b8eb0ff9e50c05945a2dc687300f362e
Jun 02 04:45:25.279261 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.279192    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7494f5fddc-msbxk"]
Jun 02 04:45:25.281746 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:45:25.281722    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17dfd930_9652_4845_a83a_9f40ea622fcd.slice/crio-41b93200f806791c1d0724add81993f3c69f49f5cf92afbe525b63093d89717c WatchSource:0}: Error finding container 41b93200f806791c1d0724add81993f3c69f49f5cf92afbe525b63093d89717c: Status 404 returned error can't find the container with id 41b93200f806791c1d0724add81993f3c69f49f5cf92afbe525b63093d89717c
Jun 02 04:45:25.889264 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.889225    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7494f5fddc-msbxk" event={"ID":"17dfd930-9652-4845-a83a-9f40ea622fcd","Type":"ContainerStarted","Data":"35a50bf8898465267e0e39da6b65a845fbc987629ee7be026fdb96dac4d5b0d4"}
Jun 02 04:45:25.889264 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.889267    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7494f5fddc-msbxk" event={"ID":"17dfd930-9652-4845-a83a-9f40ea622fcd","Type":"ContainerStarted","Data":"41b93200f806791c1d0724add81993f3c69f49f5cf92afbe525b63093d89717c"}
Jun 02 04:45:25.889771 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.889348    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:25.890518 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.890496    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rbpq7" event={"ID":"eb5b0c03-aba6-4531-a125-377ffc9f68e4","Type":"ContainerStarted","Data":"500ec2b50c27f10332cc49404aabd7747d32b365b29b78ee88e178ba8430708f"}
Jun 02 04:45:25.890617 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.890520    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rbpq7" event={"ID":"eb5b0c03-aba6-4531-a125-377ffc9f68e4","Type":"ContainerStarted","Data":"0fb0e5f3323f64f32d7d6ccd9b2d4069b8eb0ff9e50c05945a2dc687300f362e"}
Jun 02 04:45:25.911584 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:25.911550    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7494f5fddc-msbxk" podStartSLOduration=1.911537007 podStartE2EDuration="1.911537007s" podCreationTimestamp="2026-06-02 04:45:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 04:45:25.909895236 +0000 UTC m=+172.950826199" watchObservedRunningTime="2026-06-02 04:45:25.911537007 +0000 UTC m=+172.952467979"
Jun 02 04:45:26.871003 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:26.870982    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zlpr5"
Jun 02 04:45:26.897169 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:26.897143    2577 generic.go:358] "Generic (PLEG): container finished" podID="0477b761-d77b-460d-b073-05e27aebdffe" containerID="bbb0bd044106b54d80e3a1018e84f2c0731a285dcd8b8d4651e06e45b4a1fd20" exitCode=255
Jun 02 04:45:26.897544 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:26.897242    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d458f69fb-jwbd5" event={"ID":"0477b761-d77b-460d-b073-05e27aebdffe","Type":"ContainerDied","Data":"bbb0bd044106b54d80e3a1018e84f2c0731a285dcd8b8d4651e06e45b4a1fd20"}
Jun 02 04:45:26.897634 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:26.897620    2577 scope.go:117] "RemoveContainer" containerID="bbb0bd044106b54d80e3a1018e84f2c0731a285dcd8b8d4651e06e45b4a1fd20"
Jun 02 04:45:26.899169 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:26.899147    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rbpq7" event={"ID":"eb5b0c03-aba6-4531-a125-377ffc9f68e4","Type":"ContainerStarted","Data":"7edc00cbf1532cbfcde8044025ec3e4da3462dd0f1ca4d3d8615153635427d90"}
Jun 02 04:45:27.903586 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:27.903510    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d458f69fb-jwbd5" event={"ID":"0477b761-d77b-460d-b073-05e27aebdffe","Type":"ContainerStarted","Data":"cc5bc757c0552f05ad4ba2502dea17eda2236815e4ceaab508c6aca763e85c90"}
Jun 02 04:45:27.905310 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:27.905289    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rbpq7" event={"ID":"eb5b0c03-aba6-4531-a125-377ffc9f68e4","Type":"ContainerStarted","Data":"ab12baa0c082453bda5d19ba01cc7d1719b9b540467f57af6bd86ccf532fbcb1"}
Jun 02 04:45:27.941910 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:27.941873    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rbpq7" podStartSLOduration=1.6393709300000001 podStartE2EDuration="3.941862082s" podCreationTimestamp="2026-06-02 04:45:24 +0000 UTC" firstStartedPulling="2026-06-02 04:45:25.220674946 +0000 UTC m=+172.261605896" lastFinishedPulling="2026-06-02 04:45:27.523166095 +0000 UTC m=+174.564097048" observedRunningTime="2026-06-02 04:45:27.941585983 +0000 UTC m=+174.982517180" watchObservedRunningTime="2026-06-02 04:45:27.941862082 +0000 UTC m=+174.982793054"
Jun 02 04:45:36.286958 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.286918    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tcvt9"]
Jun 02 04:45:36.292154 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.292139    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.294957 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.294936    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jun 02 04:45:36.295084 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.294968    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jun 02 04:45:36.295982 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.295958    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xw5gq\""
Jun 02 04:45:36.296124 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.296059    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jun 02 04:45:36.296124 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.296057    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jun 02 04:45:36.296124 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.296075    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jun 02 04:45:36.296124 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.296075    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jun 02 04:45:36.376807 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.376784    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-accelerators-collector-config\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.376931 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.376837    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.376931 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.376906    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-tls\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.377043 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.376944    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-wtmp\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.377043 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.376995    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9600a15e-a21e-4e02-a697-4aae0cfe34e1-metrics-client-ca\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.377043 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.377022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9600a15e-a21e-4e02-a697-4aae0cfe34e1-sys\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.377154 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.377049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-textfile\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.377154 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.377104    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mklrw\" (UniqueName: \"kubernetes.io/projected/9600a15e-a21e-4e02-a697-4aae0cfe34e1-kube-api-access-mklrw\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.377213 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.377160    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9600a15e-a21e-4e02-a697-4aae0cfe34e1-root\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.477687 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.477663    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-tls\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.477781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.477691    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-wtmp\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.477856 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:45:36.477829    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jun 02 04:45:36.477856 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.477841    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9600a15e-a21e-4e02-a697-4aae0cfe34e1-metrics-client-ca\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.477961 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.477853    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-wtmp\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.477961 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.477885    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9600a15e-a21e-4e02-a697-4aae0cfe34e1-sys\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.477961 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.477904    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9600a15e-a21e-4e02-a697-4aae0cfe34e1-sys\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.477961 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.477915    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-textfile\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.477961 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:45:36.477934    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-tls podName:9600a15e-a21e-4e02-a697-4aae0cfe34e1 nodeName:}" failed. No retries permitted until 2026-06-02 04:45:36.977919549 +0000 UTC m=+184.018850499 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-tls") pod "node-exporter-tcvt9" (UID: "9600a15e-a21e-4e02-a697-4aae0cfe34e1") : secret "node-exporter-tls" not found
Jun 02 04:45:36.478185 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.477971    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mklrw\" (UniqueName: \"kubernetes.io/projected/9600a15e-a21e-4e02-a697-4aae0cfe34e1-kube-api-access-mklrw\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.478185 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.478001    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9600a15e-a21e-4e02-a697-4aae0cfe34e1-root\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.478185 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.478033    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-accelerators-collector-config\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.478185 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.478072    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.478185 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.478107    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9600a15e-a21e-4e02-a697-4aae0cfe34e1-root\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.478422 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.478271    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-textfile\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.478642 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.478622    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9600a15e-a21e-4e02-a697-4aae0cfe34e1-metrics-client-ca\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.478695 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.478623    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-accelerators-collector-config\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.480348 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.480299    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.488050 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.488024    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mklrw\" (UniqueName: \"kubernetes.io/projected/9600a15e-a21e-4e02-a697-4aae0cfe34e1-kube-api-access-mklrw\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.982397 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.982370    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-tls\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:36.984606 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:36.984578    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9600a15e-a21e-4e02-a697-4aae0cfe34e1-node-exporter-tls\") pod \"node-exporter-tcvt9\" (UID: \"9600a15e-a21e-4e02-a697-4aae0cfe34e1\") " pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:37.200194 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:37.200162    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tcvt9"
Jun 02 04:45:37.208401 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:45:37.208375    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9600a15e_a21e_4e02_a697_4aae0cfe34e1.slice/crio-ffa66196d69405b1ac7b1c3b332f282b885a01b2496a5550c0b353c75fee49d7 WatchSource:0}: Error finding container ffa66196d69405b1ac7b1c3b332f282b885a01b2496a5550c0b353c75fee49d7: Status 404 returned error can't find the container with id ffa66196d69405b1ac7b1c3b332f282b885a01b2496a5550c0b353c75fee49d7
Jun 02 04:45:37.929827 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:37.929784    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tcvt9" event={"ID":"9600a15e-a21e-4e02-a697-4aae0cfe34e1","Type":"ContainerStarted","Data":"ffa66196d69405b1ac7b1c3b332f282b885a01b2496a5550c0b353c75fee49d7"}
Jun 02 04:45:38.933157 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:38.933125    2577 generic.go:358] "Generic (PLEG): container finished" podID="9600a15e-a21e-4e02-a697-4aae0cfe34e1" containerID="ff5a37ecdf82c3560b3efaf49eba2ddbf9b6b4df1c653e373300056d539a036d" exitCode=0
Jun 02 04:45:38.933495 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:38.933189    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tcvt9" event={"ID":"9600a15e-a21e-4e02-a697-4aae0cfe34e1","Type":"ContainerDied","Data":"ff5a37ecdf82c3560b3efaf49eba2ddbf9b6b4df1c653e373300056d539a036d"}
Jun 02 04:45:39.937454 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:39.937419    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tcvt9" event={"ID":"9600a15e-a21e-4e02-a697-4aae0cfe34e1","Type":"ContainerStarted","Data":"6ec1097eba5a4b878d164ba04974bc08e981e75f9b5388bbc24211db9ef35f51"}
Jun 02 04:45:39.937797 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:39.937461    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tcvt9" event={"ID":"9600a15e-a21e-4e02-a697-4aae0cfe34e1","Type":"ContainerStarted","Data":"2542fbade108baf150898f6147338f2de1787376c7320bdc3b3f317fa36b409a"}
Jun 02 04:45:39.959245 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:39.959202    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tcvt9" podStartSLOduration=3.085411026 podStartE2EDuration="3.959190961s" podCreationTimestamp="2026-06-02 04:45:36 +0000 UTC" firstStartedPulling="2026-06-02 04:45:37.210154092 +0000 UTC m=+184.251085045" lastFinishedPulling="2026-06-02 04:45:38.083934031 +0000 UTC m=+185.124864980" observedRunningTime="2026-06-02 04:45:39.957909968 +0000 UTC m=+186.998840942" watchObservedRunningTime="2026-06-02 04:45:39.959190961 +0000 UTC m=+187.000121932"
Jun 02 04:45:40.596210 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.596186    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-8578879b87-5w6pf"]
Jun 02 04:45:40.599110 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.599093    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.603048 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.603026    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jun 02 04:45:40.603157 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.603067    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-8nvbh\""
Jun 02 04:45:40.603157 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.603026    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jun 02 04:45:40.603157 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.603026    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-eaorcqpopc987\""
Jun 02 04:45:40.603282 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.603157    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jun 02 04:45:40.603321 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.603298    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jun 02 04:45:40.608602 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.608581    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8578879b87-5w6pf"]
Jun 02 04:45:40.709698 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.709676    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdf23091-58b3-4d72-aa45-3659a2c2e924-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.709797 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.709711    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bdf23091-58b3-4d72-aa45-3659a2c2e924-secret-metrics-server-tls\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.709797 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.709731    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn9fz\" (UniqueName: \"kubernetes.io/projected/bdf23091-58b3-4d72-aa45-3659a2c2e924-kube-api-access-xn9fz\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.709797 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.709756    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bdf23091-58b3-4d72-aa45-3659a2c2e924-audit-log\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.709934 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.709794    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bdf23091-58b3-4d72-aa45-3659a2c2e924-metrics-server-audit-profiles\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.709934 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.709872    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bdf23091-58b3-4d72-aa45-3659a2c2e924-secret-metrics-server-client-certs\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.709934 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.709924    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bdf23091-58b3-4d72-aa45-3659a2c2e924-client-ca-bundle\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.810782 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.810761    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bdf23091-58b3-4d72-aa45-3659a2c2e924-secret-metrics-server-client-certs\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.810894 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.810798    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bdf23091-58b3-4d72-aa45-3659a2c2e924-client-ca-bundle\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.810894 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.810841    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdf23091-58b3-4d72-aa45-3659a2c2e924-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.810894 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.810865    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bdf23091-58b3-4d72-aa45-3659a2c2e924-secret-metrics-server-tls\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.810894 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.810884    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xn9fz\" (UniqueName: \"kubernetes.io/projected/bdf23091-58b3-4d72-aa45-3659a2c2e924-kube-api-access-xn9fz\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.811071 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.810908    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bdf23091-58b3-4d72-aa45-3659a2c2e924-audit-log\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.811071 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.810924    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bdf23091-58b3-4d72-aa45-3659a2c2e924-metrics-server-audit-profiles\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.811404 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.811376    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bdf23091-58b3-4d72-aa45-3659a2c2e924-audit-log\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.811692 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.811670    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdf23091-58b3-4d72-aa45-3659a2c2e924-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.811920 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.811901    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bdf23091-58b3-4d72-aa45-3659a2c2e924-metrics-server-audit-profiles\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.813169 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.813149    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bdf23091-58b3-4d72-aa45-3659a2c2e924-client-ca-bundle\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.813255 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.813213    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bdf23091-58b3-4d72-aa45-3659a2c2e924-secret-metrics-server-tls\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.813298 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.813278    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bdf23091-58b3-4d72-aa45-3659a2c2e924-secret-metrics-server-client-certs\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.819185 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.819167    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn9fz\" (UniqueName: \"kubernetes.io/projected/bdf23091-58b3-4d72-aa45-3659a2c2e924-kube-api-access-xn9fz\") pod \"metrics-server-8578879b87-5w6pf\" (UID: \"bdf23091-58b3-4d72-aa45-3659a2c2e924\") " pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:40.908315 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:40.908266    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:45:41.034228 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:41.034201    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8578879b87-5w6pf"]
Jun 02 04:45:41.036983 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:45:41.036954    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdf23091_58b3_4d72_aa45_3659a2c2e924.slice/crio-202074adacbdd1efd37e577f1b1cba56500261a7967d74765b2690506f684187 WatchSource:0}: Error finding container 202074adacbdd1efd37e577f1b1cba56500261a7967d74765b2690506f684187: Status 404 returned error can't find the container with id 202074adacbdd1efd37e577f1b1cba56500261a7967d74765b2690506f684187
Jun 02 04:45:41.944384 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:41.944348    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8578879b87-5w6pf" event={"ID":"bdf23091-58b3-4d72-aa45-3659a2c2e924","Type":"ContainerStarted","Data":"202074adacbdd1efd37e577f1b1cba56500261a7967d74765b2690506f684187"}
Jun 02 04:45:42.948729 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:42.948694    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8578879b87-5w6pf" event={"ID":"bdf23091-58b3-4d72-aa45-3659a2c2e924","Type":"ContainerStarted","Data":"b660cb60bde43605e824380640fd2b3e8f18c1dd66f300e33a97c52a0125f21d"}
Jun 02 04:45:42.974073 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:42.974028    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-8578879b87-5w6pf" podStartSLOduration=1.6925114730000002 podStartE2EDuration="2.974016011s" podCreationTimestamp="2026-06-02 04:45:40 +0000 UTC" firstStartedPulling="2026-06-02 04:45:41.038766934 +0000 UTC m=+188.079697884" lastFinishedPulling="2026-06-02 04:45:42.320271463 +0000 UTC m=+189.361202422" observedRunningTime="2026-06-02 04:45:42.973680149 +0000 UTC m=+190.014611121" watchObservedRunningTime="2026-06-02 04:45:42.974016011 +0000 UTC m=+190.014947113"
Jun 02 04:45:46.903861 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:46.903833    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7494f5fddc-msbxk"
Jun 02 04:45:55.795836 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.795782    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-78dc9895dc-xqpbd"]
Jun 02 04:45:55.798750 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.798734    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:55.803797 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.803770    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jun 02 04:45:55.803939 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.803846    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jun 02 04:45:55.805017 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.805001    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jun 02 04:45:55.805128 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.805117    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jun 02 04:45:55.805189 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.805179    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jun 02 04:45:55.805269 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.805250    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jun 02 04:45:55.805368 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.805357    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vhwvf\""
Jun 02 04:45:55.805759 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.805745    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jun 02 04:45:55.809058 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.809039    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jun 02 04:45:55.813280 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.813260    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78dc9895dc-xqpbd"]
Jun 02 04:45:55.916447 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.916424    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-oauth-config\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:55.916518 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.916449    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-service-ca\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:55.916518 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.916466    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj4nb\" (UniqueName: \"kubernetes.io/projected/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-kube-api-access-hj4nb\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:55.916597 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.916563    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-serving-cert\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:55.916597 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.916593    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-oauth-serving-cert\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:55.916670 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.916629    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-trusted-ca-bundle\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:55.916705 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:55.916666    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-config\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:56.017325 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.017300    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-oauth-config\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:56.017437 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.017338    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-service-ca\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:56.017437 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.017366    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hj4nb\" (UniqueName: \"kubernetes.io/projected/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-kube-api-access-hj4nb\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:56.017542 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.017437    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-serving-cert\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:56.017542 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.017485    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-oauth-serving-cert\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:56.017542 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.017530    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-trusted-ca-bundle\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:56.017686 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.017563    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-config\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:56.018246 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.018222    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-config\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:56.018327 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.018309    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-trusted-ca-bundle\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:56.018676 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.018660    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-oauth-serving-cert\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:56.018715 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.018673    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-service-ca\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:56.019676 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.019648    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-oauth-config\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:56.019891 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.019875    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-serving-cert\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:56.027715 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.027696    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj4nb\" (UniqueName: \"kubernetes.io/projected/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-kube-api-access-hj4nb\") pod \"console-78dc9895dc-xqpbd\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") " pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:56.107766 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.107719    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:45:56.236137 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.236116    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78dc9895dc-xqpbd"]
Jun 02 04:45:56.238304 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:45:56.238279    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf88bdc3c_9e42_4ed4_94b9_683bac2cadfb.slice/crio-44f596ac85ede2105e4a428a147204a52fcfc97bfcd1bc5d223f62d2f3134b7a WatchSource:0}: Error finding container 44f596ac85ede2105e4a428a147204a52fcfc97bfcd1bc5d223f62d2f3134b7a: Status 404 returned error can't find the container with id 44f596ac85ede2105e4a428a147204a52fcfc97bfcd1bc5d223f62d2f3134b7a
Jun 02 04:45:56.982674 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:56.982633    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78dc9895dc-xqpbd" event={"ID":"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb","Type":"ContainerStarted","Data":"44f596ac85ede2105e4a428a147204a52fcfc97bfcd1bc5d223f62d2f3134b7a"}
Jun 02 04:45:59.990791 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:45:59.990758    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78dc9895dc-xqpbd" event={"ID":"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb","Type":"ContainerStarted","Data":"951038220be4be418006851b5f0da36cc5ee48e22ab562771bdc38727454e904"}
Jun 02 04:46:00.018293 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:00.018246    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-78dc9895dc-xqpbd" podStartSLOduration=2.317696955 podStartE2EDuration="5.018231844s" podCreationTimestamp="2026-06-02 04:45:55 +0000 UTC" firstStartedPulling="2026-06-02 04:45:56.240030751 +0000 UTC m=+203.280961700" lastFinishedPulling="2026-06-02 04:45:58.940565624 +0000 UTC m=+205.981496589" observedRunningTime="2026-06-02 04:46:00.017273832 +0000 UTC m=+207.058204806" watchObservedRunningTime="2026-06-02 04:46:00.018231844 +0000 UTC m=+207.059162881"
Jun 02 04:46:00.909190 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:00.909147    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:46:00.909190 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:00.909203    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:46:02.508056 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:02.508013    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546" podUID="91a06157-a8f2-4bc0-b06d-a6d288988fe2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jun 02 04:46:06.108354 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:06.108323    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:46:06.108354 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:06.108357    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:46:06.112744 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:06.112725    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:46:07.009910 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:07.009885    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:46:12.507907 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:12.507862    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546" podUID="91a06157-a8f2-4bc0-b06d-a6d288988fe2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jun 02 04:46:20.913518 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:20.913490    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:46:20.917331 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:20.917307    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-8578879b87-5w6pf"
Jun 02 04:46:22.507760 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:22.507723    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546" podUID="91a06157-a8f2-4bc0-b06d-a6d288988fe2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jun 02 04:46:22.508120 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:22.507781    2577 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546"
Jun 02 04:46:22.508261 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:22.508233    2577 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"b6c37e8f03079ae9dfa993935b3901d0bfdabbb42cedd2cd946e5778eeeae8b2"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jun 02 04:46:22.508306 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:22.508292    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546" podUID="91a06157-a8f2-4bc0-b06d-a6d288988fe2" containerName="service-proxy" containerID="cri-o://b6c37e8f03079ae9dfa993935b3901d0bfdabbb42cedd2cd946e5778eeeae8b2" gracePeriod=30
Jun 02 04:46:23.047683 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:23.047656    2577 generic.go:358] "Generic (PLEG): container finished" podID="91a06157-a8f2-4bc0-b06d-a6d288988fe2" containerID="b6c37e8f03079ae9dfa993935b3901d0bfdabbb42cedd2cd946e5778eeeae8b2" exitCode=2
Jun 02 04:46:23.047853 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:23.047720    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546" event={"ID":"91a06157-a8f2-4bc0-b06d-a6d288988fe2","Type":"ContainerDied","Data":"b6c37e8f03079ae9dfa993935b3901d0bfdabbb42cedd2cd946e5778eeeae8b2"}
Jun 02 04:46:23.047853 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:23.047745    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5f75b45677-79546" event={"ID":"91a06157-a8f2-4bc0-b06d-a6d288988fe2","Type":"ContainerStarted","Data":"76bacbc78ba5cfb9a5f1c8274c0cee524e906a52c93979c7a924b8203e77d71c"}
Jun 02 04:46:45.153094 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:45.153059    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs\") pod \"network-metrics-daemon-jnktq\" (UID: \"7f26ffcc-136d-49e4-9072-626c6a31246e\") " pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:46:45.155219 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:45.155194    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f26ffcc-136d-49e4-9072-626c6a31246e-metrics-certs\") pod \"network-metrics-daemon-jnktq\" (UID: \"7f26ffcc-136d-49e4-9072-626c6a31246e\") " pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:46:45.342870 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:45.342843    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2ckqz\""
Jun 02 04:46:45.350264 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:45.350247    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jnktq"
Jun 02 04:46:45.485626 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:45.485599    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jnktq"]
Jun 02 04:46:45.488432 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:46:45.488404    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f26ffcc_136d_49e4_9072_626c6a31246e.slice/crio-2c788aa72b4057122ad845e1c254435538735a2f7c4a834c6bfe0bdb68c4250b WatchSource:0}: Error finding container 2c788aa72b4057122ad845e1c254435538735a2f7c4a834c6bfe0bdb68c4250b: Status 404 returned error can't find the container with id 2c788aa72b4057122ad845e1c254435538735a2f7c4a834c6bfe0bdb68c4250b
Jun 02 04:46:46.106374 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:46.106336    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jnktq" event={"ID":"7f26ffcc-136d-49e4-9072-626c6a31246e","Type":"ContainerStarted","Data":"2c788aa72b4057122ad845e1c254435538735a2f7c4a834c6bfe0bdb68c4250b"}
Jun 02 04:46:47.110885 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:47.110846    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jnktq" event={"ID":"7f26ffcc-136d-49e4-9072-626c6a31246e","Type":"ContainerStarted","Data":"b188b764bc0998c2a09668a3cc35f7b0e0064acd2c486412bfa7e46d6ce9b47a"}
Jun 02 04:46:47.110885 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:47.110889    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jnktq" event={"ID":"7f26ffcc-136d-49e4-9072-626c6a31246e","Type":"ContainerStarted","Data":"d661ef6d1e4a6105dda304268abce9303b1e437934799d9b8a998fbe07a7006e"}
Jun 02 04:46:47.142479 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:46:47.142413    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jnktq" podStartSLOduration=253.27846231 podStartE2EDuration="4m14.142395534s" podCreationTimestamp="2026-06-02 04:42:33 +0000 UTC" firstStartedPulling="2026-06-02 04:46:45.490661213 +0000 UTC m=+252.531592163" lastFinishedPulling="2026-06-02 04:46:46.354594432 +0000 UTC m=+253.395525387" observedRunningTime="2026-06-02 04:46:47.140163505 +0000 UTC m=+254.181094477" watchObservedRunningTime="2026-06-02 04:46:47.142395534 +0000 UTC m=+254.183326508"
Jun 02 04:47:16.969611 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:16.969580    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78dc9895dc-xqpbd"]
Jun 02 04:47:33.343634 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:33.343606    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 04:47:33.346377 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:33.346357    2577 kubelet.go:1628] "Image garbage collection succeeded"
Jun 02 04:47:33.346483 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:33.346380    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 04:47:41.988761 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:41.988719    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-78dc9895dc-xqpbd" podUID="f88bdc3c-9e42-4ed4-94b9-683bac2cadfb" containerName="console" containerID="cri-o://951038220be4be418006851b5f0da36cc5ee48e22ab562771bdc38727454e904" gracePeriod=15
Jun 02 04:47:42.225624 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.225603    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-78dc9895dc-xqpbd_f88bdc3c-9e42-4ed4-94b9-683bac2cadfb/console/0.log"
Jun 02 04:47:42.225732 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.225673    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:47:42.244630 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.244576    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-78dc9895dc-xqpbd_f88bdc3c-9e42-4ed4-94b9-683bac2cadfb/console/0.log"
Jun 02 04:47:42.244630 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.244619    2577 generic.go:358] "Generic (PLEG): container finished" podID="f88bdc3c-9e42-4ed4-94b9-683bac2cadfb" containerID="951038220be4be418006851b5f0da36cc5ee48e22ab562771bdc38727454e904" exitCode=2
Jun 02 04:47:42.244762 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.244671    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78dc9895dc-xqpbd"
Jun 02 04:47:42.244762 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.244677    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78dc9895dc-xqpbd" event={"ID":"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb","Type":"ContainerDied","Data":"951038220be4be418006851b5f0da36cc5ee48e22ab562771bdc38727454e904"}
Jun 02 04:47:42.244762 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.244719    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78dc9895dc-xqpbd" event={"ID":"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb","Type":"ContainerDied","Data":"44f596ac85ede2105e4a428a147204a52fcfc97bfcd1bc5d223f62d2f3134b7a"}
Jun 02 04:47:42.244762 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.244739    2577 scope.go:117] "RemoveContainer" containerID="951038220be4be418006851b5f0da36cc5ee48e22ab562771bdc38727454e904"
Jun 02 04:47:42.252299 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.252283    2577 scope.go:117] "RemoveContainer" containerID="951038220be4be418006851b5f0da36cc5ee48e22ab562771bdc38727454e904"
Jun 02 04:47:42.252526 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:47:42.252508    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"951038220be4be418006851b5f0da36cc5ee48e22ab562771bdc38727454e904\": container with ID starting with 951038220be4be418006851b5f0da36cc5ee48e22ab562771bdc38727454e904 not found: ID does not exist" containerID="951038220be4be418006851b5f0da36cc5ee48e22ab562771bdc38727454e904"
Jun 02 04:47:42.252575 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.252534    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"951038220be4be418006851b5f0da36cc5ee48e22ab562771bdc38727454e904"} err="failed to get container status \"951038220be4be418006851b5f0da36cc5ee48e22ab562771bdc38727454e904\": rpc error: code = NotFound desc = could not find container \"951038220be4be418006851b5f0da36cc5ee48e22ab562771bdc38727454e904\": container with ID starting with 951038220be4be418006851b5f0da36cc5ee48e22ab562771bdc38727454e904 not found: ID does not exist"
Jun 02 04:47:42.334443 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.334424    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hj4nb\" (UniqueName: \"kubernetes.io/projected/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-kube-api-access-hj4nb\") pod \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") "
Jun 02 04:47:42.334552 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.334454    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-service-ca\") pod \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") "
Jun 02 04:47:42.334552 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.334482    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-oauth-config\") pod \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") "
Jun 02 04:47:42.334552 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.334511    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-trusted-ca-bundle\") pod \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") "
Jun 02 04:47:42.334552 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.334536    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-config\") pod \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") "
Jun 02 04:47:42.334773 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.334574    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-serving-cert\") pod \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") "
Jun 02 04:47:42.334773 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.334616    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-oauth-serving-cert\") pod \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\" (UID: \"f88bdc3c-9e42-4ed4-94b9-683bac2cadfb\") "
Jun 02 04:47:42.334899 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.334881    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-service-ca" (OuterVolumeSpecName: "service-ca") pod "f88bdc3c-9e42-4ed4-94b9-683bac2cadfb" (UID: "f88bdc3c-9e42-4ed4-94b9-683bac2cadfb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 04:47:42.335011 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.334984    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f88bdc3c-9e42-4ed4-94b9-683bac2cadfb" (UID: "f88bdc3c-9e42-4ed4-94b9-683bac2cadfb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 04:47:42.335070 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.335005    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-config" (OuterVolumeSpecName: "console-config") pod "f88bdc3c-9e42-4ed4-94b9-683bac2cadfb" (UID: "f88bdc3c-9e42-4ed4-94b9-683bac2cadfb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 04:47:42.335105 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.335086    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f88bdc3c-9e42-4ed4-94b9-683bac2cadfb" (UID: "f88bdc3c-9e42-4ed4-94b9-683bac2cadfb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 04:47:42.336642 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.336624    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f88bdc3c-9e42-4ed4-94b9-683bac2cadfb" (UID: "f88bdc3c-9e42-4ed4-94b9-683bac2cadfb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 04:47:42.336906 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.336888    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-kube-api-access-hj4nb" (OuterVolumeSpecName: "kube-api-access-hj4nb") pod "f88bdc3c-9e42-4ed4-94b9-683bac2cadfb" (UID: "f88bdc3c-9e42-4ed4-94b9-683bac2cadfb"). InnerVolumeSpecName "kube-api-access-hj4nb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 04:47:42.336971 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.336954    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f88bdc3c-9e42-4ed4-94b9-683bac2cadfb" (UID: "f88bdc3c-9e42-4ed4-94b9-683bac2cadfb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 04:47:42.435532 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.435500    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-oauth-serving-cert\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:47:42.435532 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.435531    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj4nb\" (UniqueName: \"kubernetes.io/projected/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-kube-api-access-hj4nb\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:47:42.435671 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.435548    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-service-ca\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:47:42.435671 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.435562    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-oauth-config\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:47:42.435671 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.435575    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-trusted-ca-bundle\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:47:42.435671 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.435587    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-config\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:47:42.435671 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.435600    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb-console-serving-cert\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:47:42.565038 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.565013    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78dc9895dc-xqpbd"]
Jun 02 04:47:42.569010 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:42.568991    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78dc9895dc-xqpbd"]
Jun 02 04:47:43.443449 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:47:43.443414    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f88bdc3c-9e42-4ed4-94b9-683bac2cadfb" path="/var/lib/kubelet/pods/f88bdc3c-9e42-4ed4-94b9-683bac2cadfb/volumes"
Jun 02 04:48:59.451637 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.451605    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-hhdkx"]
Jun 02 04:48:59.452083 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.451836    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f88bdc3c-9e42-4ed4-94b9-683bac2cadfb" containerName="console"
Jun 02 04:48:59.452083 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.451847    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f88bdc3c-9e42-4ed4-94b9-683bac2cadfb" containerName="console"
Jun 02 04:48:59.452083 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.451897    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="f88bdc3c-9e42-4ed4-94b9-683bac2cadfb" containerName="console"
Jun 02 04:48:59.454428 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.454412    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhdkx"
Jun 02 04:48:59.456873 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.456856    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jun 02 04:48:59.463540 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.463517    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hhdkx"]
Jun 02 04:48:59.536294 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.536272    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c861fa7b-2da0-4681-99b9-9d73c9469d51-dbus\") pod \"global-pull-secret-syncer-hhdkx\" (UID: \"c861fa7b-2da0-4681-99b9-9d73c9469d51\") " pod="kube-system/global-pull-secret-syncer-hhdkx"
Jun 02 04:48:59.536398 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.536317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c861fa7b-2da0-4681-99b9-9d73c9469d51-original-pull-secret\") pod \"global-pull-secret-syncer-hhdkx\" (UID: \"c861fa7b-2da0-4681-99b9-9d73c9469d51\") " pod="kube-system/global-pull-secret-syncer-hhdkx"
Jun 02 04:48:59.536398 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.536366    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c861fa7b-2da0-4681-99b9-9d73c9469d51-kubelet-config\") pod \"global-pull-secret-syncer-hhdkx\" (UID: \"c861fa7b-2da0-4681-99b9-9d73c9469d51\") " pod="kube-system/global-pull-secret-syncer-hhdkx"
Jun 02 04:48:59.637431 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.637411    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c861fa7b-2da0-4681-99b9-9d73c9469d51-dbus\") pod \"global-pull-secret-syncer-hhdkx\" (UID: \"c861fa7b-2da0-4681-99b9-9d73c9469d51\") " pod="kube-system/global-pull-secret-syncer-hhdkx"
Jun 02 04:48:59.637542 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.637455    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c861fa7b-2da0-4681-99b9-9d73c9469d51-original-pull-secret\") pod \"global-pull-secret-syncer-hhdkx\" (UID: \"c861fa7b-2da0-4681-99b9-9d73c9469d51\") " pod="kube-system/global-pull-secret-syncer-hhdkx"
Jun 02 04:48:59.637542 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.637486    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c861fa7b-2da0-4681-99b9-9d73c9469d51-kubelet-config\") pod \"global-pull-secret-syncer-hhdkx\" (UID: \"c861fa7b-2da0-4681-99b9-9d73c9469d51\") " pod="kube-system/global-pull-secret-syncer-hhdkx"
Jun 02 04:48:59.637614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.637558    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c861fa7b-2da0-4681-99b9-9d73c9469d51-kubelet-config\") pod \"global-pull-secret-syncer-hhdkx\" (UID: \"c861fa7b-2da0-4681-99b9-9d73c9469d51\") " pod="kube-system/global-pull-secret-syncer-hhdkx"
Jun 02 04:48:59.637614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.637582    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c861fa7b-2da0-4681-99b9-9d73c9469d51-dbus\") pod \"global-pull-secret-syncer-hhdkx\" (UID: \"c861fa7b-2da0-4681-99b9-9d73c9469d51\") " pod="kube-system/global-pull-secret-syncer-hhdkx"
Jun 02 04:48:59.639516 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.639498    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c861fa7b-2da0-4681-99b9-9d73c9469d51-original-pull-secret\") pod \"global-pull-secret-syncer-hhdkx\" (UID: \"c861fa7b-2da0-4681-99b9-9d73c9469d51\") " pod="kube-system/global-pull-secret-syncer-hhdkx"
Jun 02 04:48:59.763331 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.763310    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hhdkx"
Jun 02 04:48:59.887740 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.887671    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hhdkx"]
Jun 02 04:48:59.891117 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:48:59.891091    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc861fa7b_2da0_4681_99b9_9d73c9469d51.slice/crio-9531294261ffa804be5a413e55671a5a2a8dd4f3ec523f99e77314b6dae7ef87 WatchSource:0}: Error finding container 9531294261ffa804be5a413e55671a5a2a8dd4f3ec523f99e77314b6dae7ef87: Status 404 returned error can't find the container with id 9531294261ffa804be5a413e55671a5a2a8dd4f3ec523f99e77314b6dae7ef87
Jun 02 04:48:59.893415 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:48:59.893395    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 04:49:00.437481 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:00.437448    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hhdkx" event={"ID":"c861fa7b-2da0-4681-99b9-9d73c9469d51","Type":"ContainerStarted","Data":"9531294261ffa804be5a413e55671a5a2a8dd4f3ec523f99e77314b6dae7ef87"}
Jun 02 04:49:04.449396 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:04.449318    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hhdkx" event={"ID":"c861fa7b-2da0-4681-99b9-9d73c9469d51","Type":"ContainerStarted","Data":"610082ddfe0ab5e7e03fd277a5a62da0ed06c3e1afec939c55b94193dea845dc"}
Jun 02 04:49:53.851334 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:53.851241    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hhdkx" podStartSLOduration=50.55805212 podStartE2EDuration="54.851226974s" podCreationTimestamp="2026-06-02 04:48:59 +0000 UTC" firstStartedPulling="2026-06-02 04:48:59.893581889 +0000 UTC m=+386.934512870" lastFinishedPulling="2026-06-02 04:49:04.186756771 +0000 UTC m=+391.227687724" observedRunningTime="2026-06-02 04:49:04.485761587 +0000 UTC m=+391.526692558" watchObservedRunningTime="2026-06-02 04:49:53.851226974 +0000 UTC m=+440.892157946"
Jun 02 04:49:53.851789 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:53.851468    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-m57jk"]
Jun 02 04:49:53.854201 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:53.854185    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-m57jk"
Jun 02 04:49:53.856905 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:53.856884    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jun 02 04:49:53.857893 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:53.857871    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jun 02 04:49:53.857966 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:53.857914    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-2hqkk\""
Jun 02 04:49:53.869054 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:53.869035    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-m57jk"]
Jun 02 04:49:53.882521 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:53.882503    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c341129d-f9e2-40fc-b2e2-ef49eba29f2c-bound-sa-token\") pod \"cert-manager-79c8d999ff-m57jk\" (UID: \"c341129d-f9e2-40fc-b2e2-ef49eba29f2c\") " pod="cert-manager/cert-manager-79c8d999ff-m57jk"
Jun 02 04:49:53.882614 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:53.882540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8gbp\" (UniqueName: \"kubernetes.io/projected/c341129d-f9e2-40fc-b2e2-ef49eba29f2c-kube-api-access-n8gbp\") pod \"cert-manager-79c8d999ff-m57jk\" (UID: \"c341129d-f9e2-40fc-b2e2-ef49eba29f2c\") " pod="cert-manager/cert-manager-79c8d999ff-m57jk"
Jun 02 04:49:53.983251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:53.983229    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n8gbp\" (UniqueName: \"kubernetes.io/projected/c341129d-f9e2-40fc-b2e2-ef49eba29f2c-kube-api-access-n8gbp\") pod \"cert-manager-79c8d999ff-m57jk\" (UID: \"c341129d-f9e2-40fc-b2e2-ef49eba29f2c\") " pod="cert-manager/cert-manager-79c8d999ff-m57jk"
Jun 02 04:49:53.983369 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:53.983276    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c341129d-f9e2-40fc-b2e2-ef49eba29f2c-bound-sa-token\") pod \"cert-manager-79c8d999ff-m57jk\" (UID: \"c341129d-f9e2-40fc-b2e2-ef49eba29f2c\") " pod="cert-manager/cert-manager-79c8d999ff-m57jk"
Jun 02 04:49:53.992658 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:53.992631    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c341129d-f9e2-40fc-b2e2-ef49eba29f2c-bound-sa-token\") pod \"cert-manager-79c8d999ff-m57jk\" (UID: \"c341129d-f9e2-40fc-b2e2-ef49eba29f2c\") " pod="cert-manager/cert-manager-79c8d999ff-m57jk"
Jun 02 04:49:53.992822 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:53.992793    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8gbp\" (UniqueName: \"kubernetes.io/projected/c341129d-f9e2-40fc-b2e2-ef49eba29f2c-kube-api-access-n8gbp\") pod \"cert-manager-79c8d999ff-m57jk\" (UID: \"c341129d-f9e2-40fc-b2e2-ef49eba29f2c\") " pod="cert-manager/cert-manager-79c8d999ff-m57jk"
Jun 02 04:49:54.163110 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.163051    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-m57jk"
Jun 02 04:49:54.220144 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.219840    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-bs2rj"]
Jun 02 04:49:54.224118 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.224061    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-bs2rj"
Jun 02 04:49:54.226865 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.226845    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jun 02 04:49:54.227085 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.227063    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-k7ndk\""
Jun 02 04:49:54.227174 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.227095    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jun 02 04:49:54.234121 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.234099    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-bs2rj"]
Jun 02 04:49:54.284258 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.284185    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-m57jk"]
Jun 02 04:49:54.285127 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.285107    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4473fcda-386d-4aef-a12a-1a7dab6fc1a9-tmp\") pod \"openshift-lws-operator-bfc7f696d-bs2rj\" (UID: \"4473fcda-386d-4aef-a12a-1a7dab6fc1a9\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-bs2rj"
Jun 02 04:49:54.285169 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.285139    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h76f4\" (UniqueName: \"kubernetes.io/projected/4473fcda-386d-4aef-a12a-1a7dab6fc1a9-kube-api-access-h76f4\") pod \"openshift-lws-operator-bfc7f696d-bs2rj\" (UID: \"4473fcda-386d-4aef-a12a-1a7dab6fc1a9\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-bs2rj"
Jun 02 04:49:54.286775 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:49:54.286746    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc341129d_f9e2_40fc_b2e2_ef49eba29f2c.slice/crio-8cfe3cb1c695c79769184264a4b4ebce45ab91cfddace1ff43253d1ff87fdbc4 WatchSource:0}: Error finding container 8cfe3cb1c695c79769184264a4b4ebce45ab91cfddace1ff43253d1ff87fdbc4: Status 404 returned error can't find the container with id 8cfe3cb1c695c79769184264a4b4ebce45ab91cfddace1ff43253d1ff87fdbc4
Jun 02 04:49:54.385720 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.385694    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4473fcda-386d-4aef-a12a-1a7dab6fc1a9-tmp\") pod \"openshift-lws-operator-bfc7f696d-bs2rj\" (UID: \"4473fcda-386d-4aef-a12a-1a7dab6fc1a9\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-bs2rj"
Jun 02 04:49:54.385803 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.385724    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h76f4\" (UniqueName: \"kubernetes.io/projected/4473fcda-386d-4aef-a12a-1a7dab6fc1a9-kube-api-access-h76f4\") pod \"openshift-lws-operator-bfc7f696d-bs2rj\" (UID: \"4473fcda-386d-4aef-a12a-1a7dab6fc1a9\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-bs2rj"
Jun 02 04:49:54.386026 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.386009    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4473fcda-386d-4aef-a12a-1a7dab6fc1a9-tmp\") pod \"openshift-lws-operator-bfc7f696d-bs2rj\" (UID: \"4473fcda-386d-4aef-a12a-1a7dab6fc1a9\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-bs2rj"
Jun 02 04:49:54.393711 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.393693    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h76f4\" (UniqueName: \"kubernetes.io/projected/4473fcda-386d-4aef-a12a-1a7dab6fc1a9-kube-api-access-h76f4\") pod \"openshift-lws-operator-bfc7f696d-bs2rj\" (UID: \"4473fcda-386d-4aef-a12a-1a7dab6fc1a9\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-bs2rj"
Jun 02 04:49:54.533999 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.533977    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-bs2rj"
Jun 02 04:49:54.559611 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.559581    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-m57jk" event={"ID":"c341129d-f9e2-40fc-b2e2-ef49eba29f2c","Type":"ContainerStarted","Data":"8cfe3cb1c695c79769184264a4b4ebce45ab91cfddace1ff43253d1ff87fdbc4"}
Jun 02 04:49:54.654150 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:54.654022    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-bs2rj"]
Jun 02 04:49:54.656544 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:49:54.656517    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4473fcda_386d_4aef_a12a_1a7dab6fc1a9.slice/crio-d8f1e12dc4c5b507117ddc87d67caa2850b74bf65a6e6b3d1d8719172537408e WatchSource:0}: Error finding container d8f1e12dc4c5b507117ddc87d67caa2850b74bf65a6e6b3d1d8719172537408e: Status 404 returned error can't find the container with id d8f1e12dc4c5b507117ddc87d67caa2850b74bf65a6e6b3d1d8719172537408e
Jun 02 04:49:55.563857 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:55.563803    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-bs2rj" event={"ID":"4473fcda-386d-4aef-a12a-1a7dab6fc1a9","Type":"ContainerStarted","Data":"d8f1e12dc4c5b507117ddc87d67caa2850b74bf65a6e6b3d1d8719172537408e"}
Jun 02 04:49:58.573021 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:58.572991    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-m57jk" event={"ID":"c341129d-f9e2-40fc-b2e2-ef49eba29f2c","Type":"ContainerStarted","Data":"dc18457779bd55f2976267d987ed109187866ccd6de84a6c291aebd647dabc89"}
Jun 02 04:49:58.574367 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:58.574346    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-bs2rj" event={"ID":"4473fcda-386d-4aef-a12a-1a7dab6fc1a9","Type":"ContainerStarted","Data":"bfdd35f4a9b7943eaedb7f203ac30eee26da725e35ecc8dc4f2780e108ebfc5d"}
Jun 02 04:49:58.589562 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:58.589517    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-m57jk" podStartSLOduration=1.866318704 podStartE2EDuration="5.589504231s" podCreationTimestamp="2026-06-02 04:49:53 +0000 UTC" firstStartedPulling="2026-06-02 04:49:54.288473887 +0000 UTC m=+441.329404837" lastFinishedPulling="2026-06-02 04:49:58.011659411 +0000 UTC m=+445.052590364" observedRunningTime="2026-06-02 04:49:58.589315345 +0000 UTC m=+445.630246318" watchObservedRunningTime="2026-06-02 04:49:58.589504231 +0000 UTC m=+445.630435203"
Jun 02 04:49:58.611038 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:49:58.611000    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-bs2rj" podStartSLOduration=1.200242288 podStartE2EDuration="4.610987285s" podCreationTimestamp="2026-06-02 04:49:54 +0000 UTC" firstStartedPulling="2026-06-02 04:49:54.658093688 +0000 UTC m=+441.699024637" lastFinishedPulling="2026-06-02 04:49:58.068838675 +0000 UTC m=+445.109769634" observedRunningTime="2026-06-02 04:49:58.610003729 +0000 UTC m=+445.650934701" watchObservedRunningTime="2026-06-02 04:49:58.610987285 +0000 UTC m=+445.651918257"
Jun 02 04:50:13.824345 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:13.824314    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"]
Jun 02 04:50:13.827584 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:13.827564    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:13.830396 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:13.830368    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jun 02 04:50:13.830518 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:13.830404    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jun 02 04:50:13.830518 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:13.830478    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4kpvc\""
Jun 02 04:50:13.830518 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:13.830497    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jun 02 04:50:13.830663 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:13.830503    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jun 02 04:50:13.839128 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:13.839109    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"]
Jun 02 04:50:13.926263 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:13.926244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e3270e6-5b75-4617-9e28-2e27b883c151-tmp\") pod \"opendatahub-operator-controller-manager-55fcdd494f-rww7z\" (UID: \"2e3270e6-5b75-4617-9e28-2e27b883c151\") " pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:13.926364 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:13.926276    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2e3270e6-5b75-4617-9e28-2e27b883c151-manifests\") pod \"opendatahub-operator-controller-manager-55fcdd494f-rww7z\" (UID: \"2e3270e6-5b75-4617-9e28-2e27b883c151\") " pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:13.926364 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:13.926306    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2e3270e6-5b75-4617-9e28-2e27b883c151-webhook-cert\") pod \"opendatahub-operator-controller-manager-55fcdd494f-rww7z\" (UID: \"2e3270e6-5b75-4617-9e28-2e27b883c151\") " pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:13.926451 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:13.926367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2e3270e6-5b75-4617-9e28-2e27b883c151-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55fcdd494f-rww7z\" (UID: \"2e3270e6-5b75-4617-9e28-2e27b883c151\") " pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:13.926451 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:13.926392    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sgfm\" (UniqueName: \"kubernetes.io/projected/2e3270e6-5b75-4617-9e28-2e27b883c151-kube-api-access-6sgfm\") pod \"opendatahub-operator-controller-manager-55fcdd494f-rww7z\" (UID: \"2e3270e6-5b75-4617-9e28-2e27b883c151\") " pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:14.027390 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:14.027353    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e3270e6-5b75-4617-9e28-2e27b883c151-tmp\") pod \"opendatahub-operator-controller-manager-55fcdd494f-rww7z\" (UID: \"2e3270e6-5b75-4617-9e28-2e27b883c151\") " pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:14.027501 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:14.027409    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2e3270e6-5b75-4617-9e28-2e27b883c151-manifests\") pod \"opendatahub-operator-controller-manager-55fcdd494f-rww7z\" (UID: \"2e3270e6-5b75-4617-9e28-2e27b883c151\") " pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:14.027501 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:14.027456    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2e3270e6-5b75-4617-9e28-2e27b883c151-webhook-cert\") pod \"opendatahub-operator-controller-manager-55fcdd494f-rww7z\" (UID: \"2e3270e6-5b75-4617-9e28-2e27b883c151\") " pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:14.027501 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:14.027484    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2e3270e6-5b75-4617-9e28-2e27b883c151-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55fcdd494f-rww7z\" (UID: \"2e3270e6-5b75-4617-9e28-2e27b883c151\") " pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:14.027654 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:14.027515    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6sgfm\" (UniqueName: \"kubernetes.io/projected/2e3270e6-5b75-4617-9e28-2e27b883c151-kube-api-access-6sgfm\") pod \"opendatahub-operator-controller-manager-55fcdd494f-rww7z\" (UID: \"2e3270e6-5b75-4617-9e28-2e27b883c151\") " pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:14.027873 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:14.027848    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2e3270e6-5b75-4617-9e28-2e27b883c151-manifests\") pod \"opendatahub-operator-controller-manager-55fcdd494f-rww7z\" (UID: \"2e3270e6-5b75-4617-9e28-2e27b883c151\") " pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:14.029522 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:14.029499    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e3270e6-5b75-4617-9e28-2e27b883c151-tmp\") pod \"opendatahub-operator-controller-manager-55fcdd494f-rww7z\" (UID: \"2e3270e6-5b75-4617-9e28-2e27b883c151\") " pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:14.029804 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:14.029787    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2e3270e6-5b75-4617-9e28-2e27b883c151-webhook-cert\") pod \"opendatahub-operator-controller-manager-55fcdd494f-rww7z\" (UID: \"2e3270e6-5b75-4617-9e28-2e27b883c151\") " pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:14.029993 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:14.029974    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2e3270e6-5b75-4617-9e28-2e27b883c151-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55fcdd494f-rww7z\" (UID: \"2e3270e6-5b75-4617-9e28-2e27b883c151\") " pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:14.035646 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:14.035627    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sgfm\" (UniqueName: \"kubernetes.io/projected/2e3270e6-5b75-4617-9e28-2e27b883c151-kube-api-access-6sgfm\") pod \"opendatahub-operator-controller-manager-55fcdd494f-rww7z\" (UID: \"2e3270e6-5b75-4617-9e28-2e27b883c151\") " pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:14.137382 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:14.137330    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:14.264796 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:14.264690    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"]
Jun 02 04:50:14.270021 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:50:14.269997    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e3270e6_5b75_4617_9e28_2e27b883c151.slice/crio-e6cee5c9b2567c121d756a8c7e3770658c76358f1b4f989bb607180d6673c075 WatchSource:0}: Error finding container e6cee5c9b2567c121d756a8c7e3770658c76358f1b4f989bb607180d6673c075: Status 404 returned error can't find the container with id e6cee5c9b2567c121d756a8c7e3770658c76358f1b4f989bb607180d6673c075
Jun 02 04:50:14.613075 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:14.613041    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z" event={"ID":"2e3270e6-5b75-4617-9e28-2e27b883c151","Type":"ContainerStarted","Data":"e6cee5c9b2567c121d756a8c7e3770658c76358f1b4f989bb607180d6673c075"}
Jun 02 04:50:17.623125 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:17.623090    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z" event={"ID":"2e3270e6-5b75-4617-9e28-2e27b883c151","Type":"ContainerStarted","Data":"d20ee90a61781be9eaa5cb7017c980179ddd9bc9129900fd1ad7409212835e45"}
Jun 02 04:50:18.627158 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:18.627128    2577 generic.go:358] "Generic (PLEG): container finished" podID="2e3270e6-5b75-4617-9e28-2e27b883c151" containerID="d20ee90a61781be9eaa5cb7017c980179ddd9bc9129900fd1ad7409212835e45" exitCode=0
Jun 02 04:50:18.627483 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:18.627209    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z" event={"ID":"2e3270e6-5b75-4617-9e28-2e27b883c151","Type":"ContainerDied","Data":"d20ee90a61781be9eaa5cb7017c980179ddd9bc9129900fd1ad7409212835e45"}
Jun 02 04:50:19.631525 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:19.631492    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z" event={"ID":"2e3270e6-5b75-4617-9e28-2e27b883c151","Type":"ContainerStarted","Data":"48d8605dbde53e984dbcd30251a9a299280a06fdd26fca97d3af159a95fc56a0"}
Jun 02 04:50:19.631901 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:19.631731    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:19.657337 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:19.657285    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z" podStartSLOduration=1.9887708910000002 podStartE2EDuration="6.657273976s" podCreationTimestamp="2026-06-02 04:50:13 +0000 UTC" firstStartedPulling="2026-06-02 04:50:14.271000312 +0000 UTC m=+461.311931263" lastFinishedPulling="2026-06-02 04:50:18.939503392 +0000 UTC m=+465.980434348" observedRunningTime="2026-06-02 04:50:19.656189149 +0000 UTC m=+466.697120122" watchObservedRunningTime="2026-06-02 04:50:19.657273976 +0000 UTC m=+466.698204948"
Jun 02 04:50:30.635577 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:30.635546    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-55fcdd494f-rww7z"
Jun 02 04:50:32.629260 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:32.629226    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-77644847-zjcv7"]
Jun 02 04:50:32.715659 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:32.715630    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77644847-zjcv7"]
Jun 02 04:50:32.715803 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:32.715730    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77644847-zjcv7"
Jun 02 04:50:32.719200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:32.719177    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jun 02 04:50:32.719360 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:32.719343    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jun 02 04:50:32.720185 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:32.720168    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jun 02 04:50:32.720295 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:32.720195    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jun 02 04:50:32.769183 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:32.769159    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e88f06c-6f90-4b0d-99f0-63b5ec893ed6-tmp\") pod \"kube-auth-proxy-77644847-zjcv7\" (UID: \"7e88f06c-6f90-4b0d-99f0-63b5ec893ed6\") " pod="openshift-ingress/kube-auth-proxy-77644847-zjcv7"
Jun 02 04:50:32.769274 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:32.769188    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvkwv\" (UniqueName: \"kubernetes.io/projected/7e88f06c-6f90-4b0d-99f0-63b5ec893ed6-kube-api-access-rvkwv\") pod \"kube-auth-proxy-77644847-zjcv7\" (UID: \"7e88f06c-6f90-4b0d-99f0-63b5ec893ed6\") " pod="openshift-ingress/kube-auth-proxy-77644847-zjcv7"
Jun 02 04:50:32.769274 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:32.769211    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e88f06c-6f90-4b0d-99f0-63b5ec893ed6-tls-certs\") pod \"kube-auth-proxy-77644847-zjcv7\" (UID: \"7e88f06c-6f90-4b0d-99f0-63b5ec893ed6\") " pod="openshift-ingress/kube-auth-proxy-77644847-zjcv7"
Jun 02 04:50:32.869570 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:32.869539    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e88f06c-6f90-4b0d-99f0-63b5ec893ed6-tmp\") pod \"kube-auth-proxy-77644847-zjcv7\" (UID: \"7e88f06c-6f90-4b0d-99f0-63b5ec893ed6\") " pod="openshift-ingress/kube-auth-proxy-77644847-zjcv7"
Jun 02 04:50:32.869661 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:32.869575    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rvkwv\" (UniqueName: \"kubernetes.io/projected/7e88f06c-6f90-4b0d-99f0-63b5ec893ed6-kube-api-access-rvkwv\") pod \"kube-auth-proxy-77644847-zjcv7\" (UID: \"7e88f06c-6f90-4b0d-99f0-63b5ec893ed6\") " pod="openshift-ingress/kube-auth-proxy-77644847-zjcv7"
Jun 02 04:50:32.869661 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:32.869606    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e88f06c-6f90-4b0d-99f0-63b5ec893ed6-tls-certs\") pod \"kube-auth-proxy-77644847-zjcv7\" (UID: \"7e88f06c-6f90-4b0d-99f0-63b5ec893ed6\") " pod="openshift-ingress/kube-auth-proxy-77644847-zjcv7"
Jun 02 04:50:32.871748 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:32.871723    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7e88f06c-6f90-4b0d-99f0-63b5ec893ed6-tmp\") pod \"kube-auth-proxy-77644847-zjcv7\" (UID: \"7e88f06c-6f90-4b0d-99f0-63b5ec893ed6\") " pod="openshift-ingress/kube-auth-proxy-77644847-zjcv7"
Jun 02 04:50:32.872007 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:32.871991    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e88f06c-6f90-4b0d-99f0-63b5ec893ed6-tls-certs\") pod \"kube-auth-proxy-77644847-zjcv7\" (UID: \"7e88f06c-6f90-4b0d-99f0-63b5ec893ed6\") " pod="openshift-ingress/kube-auth-proxy-77644847-zjcv7"
Jun 02 04:50:32.893741 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:32.893694    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvkwv\" (UniqueName: \"kubernetes.io/projected/7e88f06c-6f90-4b0d-99f0-63b5ec893ed6-kube-api-access-rvkwv\") pod \"kube-auth-proxy-77644847-zjcv7\" (UID: \"7e88f06c-6f90-4b0d-99f0-63b5ec893ed6\") " pod="openshift-ingress/kube-auth-proxy-77644847-zjcv7"
Jun 02 04:50:33.025024 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:33.025002    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77644847-zjcv7"
Jun 02 04:50:33.148245 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:33.148185    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-76588b95-lz5df"]
Jun 02 04:50:33.156775 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:50:33.156748    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e88f06c_6f90_4b0d_99f0_63b5ec893ed6.slice/crio-53b4ef00752965d712e7e77fad5fc07f78eea6223357177503d6bde29f8f8855 WatchSource:0}: Error finding container 53b4ef00752965d712e7e77fad5fc07f78eea6223357177503d6bde29f8f8855: Status 404 returned error can't find the container with id 53b4ef00752965d712e7e77fad5fc07f78eea6223357177503d6bde29f8f8855
Jun 02 04:50:33.179571 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:33.179543    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77644847-zjcv7"]
Jun 02 04:50:33.179665 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:33.179578    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-lz5df"]
Jun 02 04:50:33.179665 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:33.179636    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-lz5df"
Jun 02 04:50:33.182194 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:33.182175    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jun 02 04:50:33.182290 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:33.182175    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-slc6s\""
Jun 02 04:50:33.272627 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:33.272603    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3027d5df-6b2f-4eba-8261-5a2f6fb4e98e-cert\") pod \"odh-model-controller-76588b95-lz5df\" (UID: \"3027d5df-6b2f-4eba-8261-5a2f6fb4e98e\") " pod="opendatahub/odh-model-controller-76588b95-lz5df"
Jun 02 04:50:33.272754 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:33.272638    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-546ds\" (UniqueName: \"kubernetes.io/projected/3027d5df-6b2f-4eba-8261-5a2f6fb4e98e-kube-api-access-546ds\") pod \"odh-model-controller-76588b95-lz5df\" (UID: \"3027d5df-6b2f-4eba-8261-5a2f6fb4e98e\") " pod="opendatahub/odh-model-controller-76588b95-lz5df"
Jun 02 04:50:33.373354 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:33.373325    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-546ds\" (UniqueName: \"kubernetes.io/projected/3027d5df-6b2f-4eba-8261-5a2f6fb4e98e-kube-api-access-546ds\") pod \"odh-model-controller-76588b95-lz5df\" (UID: \"3027d5df-6b2f-4eba-8261-5a2f6fb4e98e\") " pod="opendatahub/odh-model-controller-76588b95-lz5df"
Jun 02 04:50:33.373470 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:33.373391    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3027d5df-6b2f-4eba-8261-5a2f6fb4e98e-cert\") pod \"odh-model-controller-76588b95-lz5df\" (UID: \"3027d5df-6b2f-4eba-8261-5a2f6fb4e98e\") " pod="opendatahub/odh-model-controller-76588b95-lz5df"
Jun 02 04:50:33.386416 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:33.386398    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jun 02 04:50:33.394060 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:50:33.394044    2577 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jun 02 04:50:33.394111 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:50:33.394093    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3027d5df-6b2f-4eba-8261-5a2f6fb4e98e-cert podName:3027d5df-6b2f-4eba-8261-5a2f6fb4e98e nodeName:}" failed. No retries permitted until 2026-06-02 04:50:33.894076279 +0000 UTC m=+480.935007230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3027d5df-6b2f-4eba-8261-5a2f6fb4e98e-cert") pod "odh-model-controller-76588b95-lz5df" (UID: "3027d5df-6b2f-4eba-8261-5a2f6fb4e98e") : secret "odh-model-controller-webhook-cert" not found
Jun 02 04:50:33.419048 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:33.418982    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-546ds\" (UniqueName: \"kubernetes.io/projected/3027d5df-6b2f-4eba-8261-5a2f6fb4e98e-kube-api-access-546ds\") pod \"odh-model-controller-76588b95-lz5df\" (UID: \"3027d5df-6b2f-4eba-8261-5a2f6fb4e98e\") " pod="opendatahub/odh-model-controller-76588b95-lz5df"
Jun 02 04:50:33.668318 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:33.668286    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77644847-zjcv7" event={"ID":"7e88f06c-6f90-4b0d-99f0-63b5ec893ed6","Type":"ContainerStarted","Data":"53b4ef00752965d712e7e77fad5fc07f78eea6223357177503d6bde29f8f8855"}
Jun 02 04:50:33.979001 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:33.978967    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3027d5df-6b2f-4eba-8261-5a2f6fb4e98e-cert\") pod \"odh-model-controller-76588b95-lz5df\" (UID: \"3027d5df-6b2f-4eba-8261-5a2f6fb4e98e\") " pod="opendatahub/odh-model-controller-76588b95-lz5df"
Jun 02 04:50:33.981268 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:33.981238    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3027d5df-6b2f-4eba-8261-5a2f6fb4e98e-cert\") pod \"odh-model-controller-76588b95-lz5df\" (UID: \"3027d5df-6b2f-4eba-8261-5a2f6fb4e98e\") " pod="opendatahub/odh-model-controller-76588b95-lz5df"
Jun 02 04:50:34.092316 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.092114    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-slc6s\""
Jun 02 04:50:34.100358 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.100332    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-lz5df"
Jun 02 04:50:34.120338 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.120305    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-6577596cd5-77thv"]
Jun 02 04:50:34.123508 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.123489    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-6577596cd5-77thv"
Jun 02 04:50:34.126487 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.126242    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jun 02 04:50:34.137931 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.137895    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-6577596cd5-77thv"]
Jun 02 04:50:34.180555 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.180524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e007367e-46a0-4676-b34c-b86649eae5ff-tls-certs\") pod \"model-serving-api-6577596cd5-77thv\" (UID: \"e007367e-46a0-4676-b34c-b86649eae5ff\") " pod="opendatahub/model-serving-api-6577596cd5-77thv"
Jun 02 04:50:34.180693 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.180567    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlgzb\" (UniqueName: \"kubernetes.io/projected/e007367e-46a0-4676-b34c-b86649eae5ff-kube-api-access-xlgzb\") pod \"model-serving-api-6577596cd5-77thv\" (UID: \"e007367e-46a0-4676-b34c-b86649eae5ff\") " pod="opendatahub/model-serving-api-6577596cd5-77thv"
Jun 02 04:50:34.272644 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.272590    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-lz5df"]
Jun 02 04:50:34.275550 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:50:34.275515    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3027d5df_6b2f_4eba_8261_5a2f6fb4e98e.slice/crio-4f68c9a4460fdc882c2f67ebc1934b66f10a506340afca189abf26acd22532c6 WatchSource:0}: Error finding container 4f68c9a4460fdc882c2f67ebc1934b66f10a506340afca189abf26acd22532c6: Status 404 returned error can't find the container with id 4f68c9a4460fdc882c2f67ebc1934b66f10a506340afca189abf26acd22532c6
Jun 02 04:50:34.281525 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.281495    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e007367e-46a0-4676-b34c-b86649eae5ff-tls-certs\") pod \"model-serving-api-6577596cd5-77thv\" (UID: \"e007367e-46a0-4676-b34c-b86649eae5ff\") " pod="opendatahub/model-serving-api-6577596cd5-77thv"
Jun 02 04:50:34.281644 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.281538    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xlgzb\" (UniqueName: \"kubernetes.io/projected/e007367e-46a0-4676-b34c-b86649eae5ff-kube-api-access-xlgzb\") pod \"model-serving-api-6577596cd5-77thv\" (UID: \"e007367e-46a0-4676-b34c-b86649eae5ff\") " pod="opendatahub/model-serving-api-6577596cd5-77thv"
Jun 02 04:50:34.284385 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.284359    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e007367e-46a0-4676-b34c-b86649eae5ff-tls-certs\") pod \"model-serving-api-6577596cd5-77thv\" (UID: \"e007367e-46a0-4676-b34c-b86649eae5ff\") " pod="opendatahub/model-serving-api-6577596cd5-77thv"
Jun 02 04:50:34.290003 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.289960    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlgzb\" (UniqueName: \"kubernetes.io/projected/e007367e-46a0-4676-b34c-b86649eae5ff-kube-api-access-xlgzb\") pod \"model-serving-api-6577596cd5-77thv\" (UID: \"e007367e-46a0-4676-b34c-b86649eae5ff\") " pod="opendatahub/model-serving-api-6577596cd5-77thv"
Jun 02 04:50:34.438936 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.438901    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-6577596cd5-77thv"
Jun 02 04:50:34.586360 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.586323    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-6577596cd5-77thv"]
Jun 02 04:50:34.589117 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:50:34.589086    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode007367e_46a0_4676_b34c_b86649eae5ff.slice/crio-5fb578498ac0c8a3060b48d6c4d176b8178c3d95569a20a3acfcfe0d358e7ef8 WatchSource:0}: Error finding container 5fb578498ac0c8a3060b48d6c4d176b8178c3d95569a20a3acfcfe0d358e7ef8: Status 404 returned error can't find the container with id 5fb578498ac0c8a3060b48d6c4d176b8178c3d95569a20a3acfcfe0d358e7ef8
Jun 02 04:50:34.673654 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.673621    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-lz5df" event={"ID":"3027d5df-6b2f-4eba-8261-5a2f6fb4e98e","Type":"ContainerStarted","Data":"4f68c9a4460fdc882c2f67ebc1934b66f10a506340afca189abf26acd22532c6"}
Jun 02 04:50:34.674739 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:34.674706    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-6577596cd5-77thv" event={"ID":"e007367e-46a0-4676-b34c-b86649eae5ff","Type":"ContainerStarted","Data":"5fb578498ac0c8a3060b48d6c4d176b8178c3d95569a20a3acfcfe0d358e7ef8"}
Jun 02 04:50:37.652785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:37.652767    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jun 02 04:50:38.687867 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:38.687831    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-lz5df" event={"ID":"3027d5df-6b2f-4eba-8261-5a2f6fb4e98e","Type":"ContainerStarted","Data":"3c45517e99dbcacd82ab08edf30f5005c328111af866e81706d63f06d4e267d9"}
Jun 02 04:50:38.688297 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:38.687951    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-76588b95-lz5df"
Jun 02 04:50:38.689054 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:38.689032    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77644847-zjcv7" event={"ID":"7e88f06c-6f90-4b0d-99f0-63b5ec893ed6","Type":"ContainerStarted","Data":"2b0a70165e78a7529168dddfb719a811191dab0211ff1ac881edd8068f8aff65"}
Jun 02 04:50:38.690278 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:38.690255    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-6577596cd5-77thv" event={"ID":"e007367e-46a0-4676-b34c-b86649eae5ff","Type":"ContainerStarted","Data":"50cbae8af7636fe00fcd312b8c786f889d438d663f8fd3b2725225a4276e3a65"}
Jun 02 04:50:38.690411 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:38.690356    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-6577596cd5-77thv"
Jun 02 04:50:38.710434 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:38.707663    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-76588b95-lz5df" podStartSLOduration=1.96733519 podStartE2EDuration="5.707646262s" podCreationTimestamp="2026-06-02 04:50:33 +0000 UTC" firstStartedPulling="2026-06-02 04:50:34.277684535 +0000 UTC m=+481.318615485" lastFinishedPulling="2026-06-02 04:50:38.017995594 +0000 UTC m=+485.058926557" observedRunningTime="2026-06-02 04:50:38.705111144 +0000 UTC m=+485.746042116" watchObservedRunningTime="2026-06-02 04:50:38.707646262 +0000 UTC m=+485.748577234"
Jun 02 04:50:38.723458 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:38.723419    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-77644847-zjcv7" podStartSLOduration=2.231677982 podStartE2EDuration="6.723408674s" podCreationTimestamp="2026-06-02 04:50:32 +0000 UTC" firstStartedPulling="2026-06-02 04:50:33.15846331 +0000 UTC m=+480.199394262" lastFinishedPulling="2026-06-02 04:50:37.650194002 +0000 UTC m=+484.691124954" observedRunningTime="2026-06-02 04:50:38.722614527 +0000 UTC m=+485.763545501" watchObservedRunningTime="2026-06-02 04:50:38.723408674 +0000 UTC m=+485.764339732"
Jun 02 04:50:38.740270 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:38.740227    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/model-serving-api-6577596cd5-77thv" podStartSLOduration=1.277427299 podStartE2EDuration="4.740215455s" podCreationTimestamp="2026-06-02 04:50:34 +0000 UTC" firstStartedPulling="2026-06-02 04:50:34.590880497 +0000 UTC m=+481.631811454" lastFinishedPulling="2026-06-02 04:50:38.053668656 +0000 UTC m=+485.094599610" observedRunningTime="2026-06-02 04:50:38.739380539 +0000 UTC m=+485.780311511" watchObservedRunningTime="2026-06-02 04:50:38.740215455 +0000 UTC m=+485.781146426"
Jun 02 04:50:44.025240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.025207    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn"]
Jun 02 04:50:44.029390 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.029357    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn"
Jun 02 04:50:44.033177 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.033153    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-bgr4m\""
Jun 02 04:50:44.033421 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.033401    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jun 02 04:50:44.033536 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.033516    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-fstgz"]
Jun 02 04:50:44.036716 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.036701    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-fstgz"
Jun 02 04:50:44.039283 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.039265    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jun 02 04:50:44.039392 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.039342    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-nzxbn\""
Jun 02 04:50:44.048409 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.048389    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn"]
Jun 02 04:50:44.066594 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.066571    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-fstgz"]
Jun 02 04:50:44.159410 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.159375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87ghp\" (UniqueName: \"kubernetes.io/projected/ae936840-9758-4f32-9197-33c2bf5e7fea-kube-api-access-87ghp\") pod \"llmisvc-controller-manager-c6b9847d7-fstgz\" (UID: \"ae936840-9758-4f32-9197-33c2bf5e7fea\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-fstgz"
Jun 02 04:50:44.159569 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.159443    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/017432b7-cfc9-4dab-8dae-4cd4b48c0479-cert\") pod \"kserve-controller-manager-7c99ffcdcd-4wxxn\" (UID: \"017432b7-cfc9-4dab-8dae-4cd4b48c0479\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn"
Jun 02 04:50:44.159569 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.159513    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae936840-9758-4f32-9197-33c2bf5e7fea-cert\") pod \"llmisvc-controller-manager-c6b9847d7-fstgz\" (UID: \"ae936840-9758-4f32-9197-33c2bf5e7fea\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-fstgz"
Jun 02 04:50:44.159569 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.159542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-469km\" (UniqueName: \"kubernetes.io/projected/017432b7-cfc9-4dab-8dae-4cd4b48c0479-kube-api-access-469km\") pod \"kserve-controller-manager-7c99ffcdcd-4wxxn\" (UID: \"017432b7-cfc9-4dab-8dae-4cd4b48c0479\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn"
Jun 02 04:50:44.260242 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.260214    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/017432b7-cfc9-4dab-8dae-4cd4b48c0479-cert\") pod \"kserve-controller-manager-7c99ffcdcd-4wxxn\" (UID: \"017432b7-cfc9-4dab-8dae-4cd4b48c0479\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn"
Jun 02 04:50:44.260421 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.260255    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae936840-9758-4f32-9197-33c2bf5e7fea-cert\") pod \"llmisvc-controller-manager-c6b9847d7-fstgz\" (UID: \"ae936840-9758-4f32-9197-33c2bf5e7fea\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-fstgz"
Jun 02 04:50:44.260421 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.260273    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-469km\" (UniqueName: \"kubernetes.io/projected/017432b7-cfc9-4dab-8dae-4cd4b48c0479-kube-api-access-469km\") pod \"kserve-controller-manager-7c99ffcdcd-4wxxn\" (UID: \"017432b7-cfc9-4dab-8dae-4cd4b48c0479\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn"
Jun 02 04:50:44.260421 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.260294    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-87ghp\" (UniqueName: \"kubernetes.io/projected/ae936840-9758-4f32-9197-33c2bf5e7fea-kube-api-access-87ghp\") pod \"llmisvc-controller-manager-c6b9847d7-fstgz\" (UID: \"ae936840-9758-4f32-9197-33c2bf5e7fea\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-fstgz"
Jun 02 04:50:44.260421 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:50:44.260362    2577 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jun 02 04:50:44.260610 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:50:44.260449    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/017432b7-cfc9-4dab-8dae-4cd4b48c0479-cert podName:017432b7-cfc9-4dab-8dae-4cd4b48c0479 nodeName:}" failed. No retries permitted until 2026-06-02 04:50:44.760426422 +0000 UTC m=+491.801357387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/017432b7-cfc9-4dab-8dae-4cd4b48c0479-cert") pod "kserve-controller-manager-7c99ffcdcd-4wxxn" (UID: "017432b7-cfc9-4dab-8dae-4cd4b48c0479") : secret "kserve-webhook-server-cert" not found
Jun 02 04:50:44.262677 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.262652    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae936840-9758-4f32-9197-33c2bf5e7fea-cert\") pod \"llmisvc-controller-manager-c6b9847d7-fstgz\" (UID: \"ae936840-9758-4f32-9197-33c2bf5e7fea\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-fstgz"
Jun 02 04:50:44.284084 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.284029    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87ghp\" (UniqueName: \"kubernetes.io/projected/ae936840-9758-4f32-9197-33c2bf5e7fea-kube-api-access-87ghp\") pod \"llmisvc-controller-manager-c6b9847d7-fstgz\" (UID: \"ae936840-9758-4f32-9197-33c2bf5e7fea\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-fstgz"
Jun 02 04:50:44.302519 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.302494    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-469km\" (UniqueName: \"kubernetes.io/projected/017432b7-cfc9-4dab-8dae-4cd4b48c0479-kube-api-access-469km\") pod \"kserve-controller-manager-7c99ffcdcd-4wxxn\" (UID: \"017432b7-cfc9-4dab-8dae-4cd4b48c0479\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn"
Jun 02 04:50:44.348230 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.348208    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-fstgz"
Jun 02 04:50:44.481270 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.481248    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-fstgz"]
Jun 02 04:50:44.481801 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:50:44.481777    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podae936840_9758_4f32_9197_33c2bf5e7fea.slice/crio-b387dca5d71696293014af9d8a1347dabcc210fb2c765065c5391c5dba2af874 WatchSource:0}: Error finding container b387dca5d71696293014af9d8a1347dabcc210fb2c765065c5391c5dba2af874: Status 404 returned error can't find the container with id b387dca5d71696293014af9d8a1347dabcc210fb2c765065c5391c5dba2af874
Jun 02 04:50:44.709779 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.709711    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-fstgz" event={"ID":"ae936840-9758-4f32-9197-33c2bf5e7fea","Type":"ContainerStarted","Data":"b387dca5d71696293014af9d8a1347dabcc210fb2c765065c5391c5dba2af874"}
Jun 02 04:50:44.764705 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.764680    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/017432b7-cfc9-4dab-8dae-4cd4b48c0479-cert\") pod \"kserve-controller-manager-7c99ffcdcd-4wxxn\" (UID: \"017432b7-cfc9-4dab-8dae-4cd4b48c0479\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn"
Jun 02 04:50:44.766950 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.766933    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/017432b7-cfc9-4dab-8dae-4cd4b48c0479-cert\") pod \"kserve-controller-manager-7c99ffcdcd-4wxxn\" (UID: \"017432b7-cfc9-4dab-8dae-4cd4b48c0479\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn"
Jun 02 04:50:44.939913 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:44.939891    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn"
Jun 02 04:50:45.077739 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:45.076623    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn"]
Jun 02 04:50:45.119036 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:50:45.119001    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod017432b7_cfc9_4dab_8dae_4cd4b48c0479.slice/crio-92e968674bbb36b81578e905139501e55328aa1cf4c744639e8b7806e275c4b0 WatchSource:0}: Error finding container 92e968674bbb36b81578e905139501e55328aa1cf4c744639e8b7806e275c4b0: Status 404 returned error can't find the container with id 92e968674bbb36b81578e905139501e55328aa1cf4c744639e8b7806e275c4b0
Jun 02 04:50:45.714893 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:45.714851    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn" event={"ID":"017432b7-cfc9-4dab-8dae-4cd4b48c0479","Type":"ContainerStarted","Data":"92e968674bbb36b81578e905139501e55328aa1cf4c744639e8b7806e275c4b0"}
Jun 02 04:50:48.656771 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.656685    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-5bb69db85-jblwh"]
Jun 02 04:50:48.666239 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.666213    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-5bb69db85-jblwh"
Jun 02 04:50:48.668788 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.668765    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\""
Jun 02 04:50:48.669122 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.668794    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-m5rkv\""
Jun 02 04:50:48.669608 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.669588    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-5bb69db85-jblwh"]
Jun 02 04:50:48.725256 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.725223    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn" event={"ID":"017432b7-cfc9-4dab-8dae-4cd4b48c0479","Type":"ContainerStarted","Data":"c92cdc702754e5d9047ec9d7cf27842640723c625a5441ee9503266a12396acf"}
Jun 02 04:50:48.725458 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.725439    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn"
Jun 02 04:50:48.726469 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.726450    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-fstgz" event={"ID":"ae936840-9758-4f32-9197-33c2bf5e7fea","Type":"ContainerStarted","Data":"18763b911a11fe1b4ff22ca0d88ff5dd2991d3ca71c6ec601edb764e32b33cad"}
Jun 02 04:50:48.726611 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.726599    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-fstgz"
Jun 02 04:50:48.760747 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.760704    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn" podStartSLOduration=2.582039361 podStartE2EDuration="5.760690221s" podCreationTimestamp="2026-06-02 04:50:43 +0000 UTC" firstStartedPulling="2026-06-02 04:50:45.120404209 +0000 UTC m=+492.161335158" lastFinishedPulling="2026-06-02 04:50:48.299055057 +0000 UTC m=+495.339986018" observedRunningTime="2026-06-02 04:50:48.75974375 +0000 UTC m=+495.800674723" watchObservedRunningTime="2026-06-02 04:50:48.760690221 +0000 UTC m=+495.801621198"
Jun 02 04:50:48.798667 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.798617    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-fstgz" podStartSLOduration=0.984619771 podStartE2EDuration="4.79860635s" podCreationTimestamp="2026-06-02 04:50:44 +0000 UTC" firstStartedPulling="2026-06-02 04:50:44.483143368 +0000 UTC m=+491.524074317" lastFinishedPulling="2026-06-02 04:50:48.297129943 +0000 UTC m=+495.338060896" observedRunningTime="2026-06-02 04:50:48.798138352 +0000 UTC m=+495.839069323" watchObservedRunningTime="2026-06-02 04:50:48.79860635 +0000 UTC m=+495.839537321"
Jun 02 04:50:48.799476 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.799454    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsxms\" (UniqueName: \"kubernetes.io/projected/3ac5a2ae-2a3b-4463-a5b5-9f091d343132-kube-api-access-vsxms\") pod \"dashboard-redirect-5bb69db85-jblwh\" (UID: \"3ac5a2ae-2a3b-4463-a5b5-9f091d343132\") " pod="opendatahub/dashboard-redirect-5bb69db85-jblwh"
Jun 02 04:50:48.799542 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.799526    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/3ac5a2ae-2a3b-4463-a5b5-9f091d343132-redirect-config\") pod \"dashboard-redirect-5bb69db85-jblwh\" (UID: \"3ac5a2ae-2a3b-4463-a5b5-9f091d343132\") " pod="opendatahub/dashboard-redirect-5bb69db85-jblwh"
Jun 02 04:50:48.900896 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.900870    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vsxms\" (UniqueName: \"kubernetes.io/projected/3ac5a2ae-2a3b-4463-a5b5-9f091d343132-kube-api-access-vsxms\") pod \"dashboard-redirect-5bb69db85-jblwh\" (UID: \"3ac5a2ae-2a3b-4463-a5b5-9f091d343132\") " pod="opendatahub/dashboard-redirect-5bb69db85-jblwh"
Jun 02 04:50:48.901107 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.901088    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/3ac5a2ae-2a3b-4463-a5b5-9f091d343132-redirect-config\") pod \"dashboard-redirect-5bb69db85-jblwh\" (UID: \"3ac5a2ae-2a3b-4463-a5b5-9f091d343132\") " pod="opendatahub/dashboard-redirect-5bb69db85-jblwh"
Jun 02 04:50:48.901710 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.901690    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/3ac5a2ae-2a3b-4463-a5b5-9f091d343132-redirect-config\") pod \"dashboard-redirect-5bb69db85-jblwh\" (UID: \"3ac5a2ae-2a3b-4463-a5b5-9f091d343132\") " pod="opendatahub/dashboard-redirect-5bb69db85-jblwh"
Jun 02 04:50:48.910764 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.910718    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsxms\" (UniqueName: \"kubernetes.io/projected/3ac5a2ae-2a3b-4463-a5b5-9f091d343132-kube-api-access-vsxms\") pod \"dashboard-redirect-5bb69db85-jblwh\" (UID: \"3ac5a2ae-2a3b-4463-a5b5-9f091d343132\") " pod="opendatahub/dashboard-redirect-5bb69db85-jblwh"
Jun 02 04:50:48.923738 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.923716    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-xqm9c"]
Jun 02 04:50:48.927035 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.927022    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xqm9c"
Jun 02 04:50:48.929723 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.929705    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-thqxv\""
Jun 02 04:50:48.929807 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.929706    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\""
Jun 02 04:50:48.930012 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.929990    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\""
Jun 02 04:50:48.937594 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.937570    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-xqm9c"]
Jun 02 04:50:48.977671 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:48.977653    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-5bb69db85-jblwh"
Jun 02 04:50:49.102080 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:49.102052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/f6955e3a-507d-4bd5-9ca4-c69369d47321-operator-config\") pod \"servicemesh-operator3-55f49c5f94-xqm9c\" (UID: \"f6955e3a-507d-4bd5-9ca4-c69369d47321\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xqm9c"
Jun 02 04:50:49.102217 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:49.102175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvnwd\" (UniqueName: \"kubernetes.io/projected/f6955e3a-507d-4bd5-9ca4-c69369d47321-kube-api-access-cvnwd\") pod \"servicemesh-operator3-55f49c5f94-xqm9c\" (UID: \"f6955e3a-507d-4bd5-9ca4-c69369d47321\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xqm9c"
Jun 02 04:50:49.102396 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:49.102375    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-5bb69db85-jblwh"]
Jun 02 04:50:49.104863 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:50:49.104834    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ac5a2ae_2a3b_4463_a5b5_9f091d343132.slice/crio-f1af38cc83a71e4c02adf39d7452c952168412ec1cbe443b93f41b47056aaa52 WatchSource:0}: Error finding container f1af38cc83a71e4c02adf39d7452c952168412ec1cbe443b93f41b47056aaa52: Status 404 returned error can't find the container with id f1af38cc83a71e4c02adf39d7452c952168412ec1cbe443b93f41b47056aaa52
Jun 02 04:50:49.202758 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:49.202694    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/f6955e3a-507d-4bd5-9ca4-c69369d47321-operator-config\") pod \"servicemesh-operator3-55f49c5f94-xqm9c\" (UID: \"f6955e3a-507d-4bd5-9ca4-c69369d47321\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xqm9c"
Jun 02 04:50:49.202758 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:49.202743    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cvnwd\" (UniqueName: \"kubernetes.io/projected/f6955e3a-507d-4bd5-9ca4-c69369d47321-kube-api-access-cvnwd\") pod \"servicemesh-operator3-55f49c5f94-xqm9c\" (UID: \"f6955e3a-507d-4bd5-9ca4-c69369d47321\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xqm9c"
Jun 02 04:50:49.205102 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:49.205084    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/f6955e3a-507d-4bd5-9ca4-c69369d47321-operator-config\") pod \"servicemesh-operator3-55f49c5f94-xqm9c\" (UID: \"f6955e3a-507d-4bd5-9ca4-c69369d47321\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xqm9c"
Jun 02 04:50:49.212462 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:49.212441    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvnwd\" (UniqueName: \"kubernetes.io/projected/f6955e3a-507d-4bd5-9ca4-c69369d47321-kube-api-access-cvnwd\") pod \"servicemesh-operator3-55f49c5f94-xqm9c\" (UID: \"f6955e3a-507d-4bd5-9ca4-c69369d47321\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-xqm9c"
Jun 02 04:50:49.236375 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:49.236350    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xqm9c"
Jun 02 04:50:49.382149 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:49.382125    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-xqm9c"]
Jun 02 04:50:49.384026 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:50:49.383994    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6955e3a_507d_4bd5_9ca4_c69369d47321.slice/crio-2c1d68f6a7a03063d821c03c9eaded4cde6522a1106152c23b5212b13a535b43 WatchSource:0}: Error finding container 2c1d68f6a7a03063d821c03c9eaded4cde6522a1106152c23b5212b13a535b43: Status 404 returned error can't find the container with id 2c1d68f6a7a03063d821c03c9eaded4cde6522a1106152c23b5212b13a535b43
Jun 02 04:50:49.695753 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:49.695722    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-76588b95-lz5df"
Jun 02 04:50:49.698931 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:49.698911    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-6577596cd5-77thv"
Jun 02 04:50:49.732193 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:49.732165    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-5bb69db85-jblwh" event={"ID":"3ac5a2ae-2a3b-4463-a5b5-9f091d343132","Type":"ContainerStarted","Data":"f1af38cc83a71e4c02adf39d7452c952168412ec1cbe443b93f41b47056aaa52"}
Jun 02 04:50:49.733379 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:49.733354    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xqm9c" event={"ID":"f6955e3a-507d-4bd5-9ca4-c69369d47321","Type":"ContainerStarted","Data":"2c1d68f6a7a03063d821c03c9eaded4cde6522a1106152c23b5212b13a535b43"}
Jun 02 04:50:52.745701 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:52.745619    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xqm9c" event={"ID":"f6955e3a-507d-4bd5-9ca4-c69369d47321","Type":"ContainerStarted","Data":"e719067e63bc83151d8da433b6fe5857ad62c055164e328562ff5ac81b3f7071"}
Jun 02 04:50:52.746155 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:52.745723    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xqm9c"
Jun 02 04:50:52.747278 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:52.747255    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-5bb69db85-jblwh" event={"ID":"3ac5a2ae-2a3b-4463-a5b5-9f091d343132","Type":"ContainerStarted","Data":"47929b8ec67b141f069799d0e681767289ffc5ca11d9170a73bdcb6fb7f13b8d"}
Jun 02 04:50:52.771770 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:52.771723    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xqm9c" podStartSLOduration=1.78753959 podStartE2EDuration="4.771710003s" podCreationTimestamp="2026-06-02 04:50:48 +0000 UTC" firstStartedPulling="2026-06-02 04:50:49.386344042 +0000 UTC m=+496.427274997" lastFinishedPulling="2026-06-02 04:50:52.370514457 +0000 UTC m=+499.411445410" observedRunningTime="2026-06-02 04:50:52.770371534 +0000 UTC m=+499.811302507" watchObservedRunningTime="2026-06-02 04:50:52.771710003 +0000 UTC m=+499.812640975"
Jun 02 04:50:52.788328 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:52.788287    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-5bb69db85-jblwh" podStartSLOduration=1.527789617 podStartE2EDuration="4.788276402s" podCreationTimestamp="2026-06-02 04:50:48 +0000 UTC" firstStartedPulling="2026-06-02 04:50:49.10606851 +0000 UTC m=+496.146999459" lastFinishedPulling="2026-06-02 04:50:52.36655529 +0000 UTC m=+499.407486244" observedRunningTime="2026-06-02 04:50:52.786784723 +0000 UTC m=+499.827715689" watchObservedRunningTime="2026-06-02 04:50:52.788276402 +0000 UTC m=+499.829207374"
Jun 02 04:50:57.708681 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.708630    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"]
Jun 02 04:50:57.712350 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.712328    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.715061 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.715037    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-vhdnk\""
Jun 02 04:50:57.715194 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.715037    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jun 02 04:50:57.722789 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.722514    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"]
Jun 02 04:50:57.874408 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.874369    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcz4x\" (UniqueName: \"kubernetes.io/projected/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-kube-api-access-dcz4x\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.874408 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.874407    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.874640 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.874426    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.874640 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.874483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.874640 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.874507    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.874640 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.874543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.874640 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.874560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.874640 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.874582    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.874640 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.874600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.975595 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.975503    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dcz4x\" (UniqueName: \"kubernetes.io/projected/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-kube-api-access-dcz4x\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.975595 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.975538    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.975595 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.975558    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.975595 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.975584    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.975925 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.975610    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.975925 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.975645    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.975925 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.975663    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.975925 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.975692    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.975925 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.975721    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.976156 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.975996    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.976156 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.976060    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.976247 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.976188    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.976303 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.976248    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.976477 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.976460    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.977926 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.977902    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.978233 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.978217    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.986077 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.986050    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:57.986316 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:57.986294    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcz4x\" (UniqueName: \"kubernetes.io/projected/158b6c5f-4896-44f2-aeba-58fcaf5a7da1-kube-api-access-dcz4x\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b5gv92\" (UID: \"158b6c5f-4896-44f2-aeba-58fcaf5a7da1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:58.026052 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:58.026031    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:50:58.161954 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:58.161927    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"]
Jun 02 04:50:58.164495 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:50:58.164466    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod158b6c5f_4896_44f2_aeba_58fcaf5a7da1.slice/crio-1be8da5393e8e101116b5cd2bc1d2ea10af6da532240ac2208e7e7742665845f WatchSource:0}: Error finding container 1be8da5393e8e101116b5cd2bc1d2ea10af6da532240ac2208e7e7742665845f: Status 404 returned error can't find the container with id 1be8da5393e8e101116b5cd2bc1d2ea10af6da532240ac2208e7e7742665845f
Jun 02 04:50:58.767398 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:50:58.767364    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92" event={"ID":"158b6c5f-4896-44f2-aeba-58fcaf5a7da1","Type":"ContainerStarted","Data":"1be8da5393e8e101116b5cd2bc1d2ea10af6da532240ac2208e7e7742665845f"}
Jun 02 04:51:00.876211 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:00.876174    2577 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jun 02 04:51:00.876502 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:00.876246    2577 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jun 02 04:51:00.876502 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:00.876275    2577 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jun 02 04:51:01.785666 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:01.785631    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92" event={"ID":"158b6c5f-4896-44f2-aeba-58fcaf5a7da1","Type":"ContainerStarted","Data":"be7732a68d963b375e15db7479cc47ba92010694bab55a94cfa3bb8dbf65578e"}
Jun 02 04:51:01.807157 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:01.807109    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92" podStartSLOduration=2.097613778 podStartE2EDuration="4.807092668s" podCreationTimestamp="2026-06-02 04:50:57 +0000 UTC" firstStartedPulling="2026-06-02 04:50:58.166407193 +0000 UTC m=+505.207338142" lastFinishedPulling="2026-06-02 04:51:00.875886078 +0000 UTC m=+507.916817032" observedRunningTime="2026-06-02 04:51:01.806748428 +0000 UTC m=+508.847679415" watchObservedRunningTime="2026-06-02 04:51:01.807092668 +0000 UTC m=+508.848023641"
Jun 02 04:51:02.026461 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:02.026425    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:51:02.031274 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:02.031248    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:51:02.788366 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:02.788336    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:51:02.789320 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:02.789302    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b5gv92"
Jun 02 04:51:03.752049 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:03.752024    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-xqm9c"
Jun 02 04:51:19.735796 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:19.735765    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-fstgz"
Jun 02 04:51:19.738940 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:19.738889    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-4wxxn"
Jun 02 04:51:33.970858 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:33.970825    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xl4db"]
Jun 02 04:51:33.990690 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:33.990655    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xl4db"]
Jun 02 04:51:33.990690 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:33.990681    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xl4db"
Jun 02 04:51:33.995763 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:33.995743    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-qxl7g\""
Jun 02 04:51:33.999883 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:33.999866    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jun 02 04:51:34.000000 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:33.999886    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jun 02 04:51:34.026719 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:34.026694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnwg6\" (UniqueName: \"kubernetes.io/projected/a1a6a35f-b865-4101-b6df-00ec03f01a0a-kube-api-access-hnwg6\") pod \"kuadrant-operator-catalog-xl4db\" (UID: \"a1a6a35f-b865-4101-b6df-00ec03f01a0a\") " pod="kuadrant-system/kuadrant-operator-catalog-xl4db"
Jun 02 04:51:34.128062 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:34.128034    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hnwg6\" (UniqueName: \"kubernetes.io/projected/a1a6a35f-b865-4101-b6df-00ec03f01a0a-kube-api-access-hnwg6\") pod \"kuadrant-operator-catalog-xl4db\" (UID: \"a1a6a35f-b865-4101-b6df-00ec03f01a0a\") " pod="kuadrant-system/kuadrant-operator-catalog-xl4db"
Jun 02 04:51:34.135534 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:34.135506    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnwg6\" (UniqueName: \"kubernetes.io/projected/a1a6a35f-b865-4101-b6df-00ec03f01a0a-kube-api-access-hnwg6\") pod \"kuadrant-operator-catalog-xl4db\" (UID: \"a1a6a35f-b865-4101-b6df-00ec03f01a0a\") " pod="kuadrant-system/kuadrant-operator-catalog-xl4db"
Jun 02 04:51:34.301908 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:34.301881    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xl4db"
Jun 02 04:51:34.331566 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:34.331541    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xl4db"]
Jun 02 04:51:34.426377 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:34.426353    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xl4db"]
Jun 02 04:51:34.428533 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:51:34.428504    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a6a35f_b865_4101_b6df_00ec03f01a0a.slice/crio-247e1b2911007af6064488e23def890d7a1d21bfd4fd8c84f8a1ae39824c35fd WatchSource:0}: Error finding container 247e1b2911007af6064488e23def890d7a1d21bfd4fd8c84f8a1ae39824c35fd: Status 404 returned error can't find the container with id 247e1b2911007af6064488e23def890d7a1d21bfd4fd8c84f8a1ae39824c35fd
Jun 02 04:51:34.544029 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:34.542724    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bsqqk"]
Jun 02 04:51:34.547678 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:34.547656    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bsqqk"
Jun 02 04:51:34.556751 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:34.555953    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bsqqk"]
Jun 02 04:51:34.632152 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:34.632124    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m46p9\" (UniqueName: \"kubernetes.io/projected/756f6c87-a8ca-4cff-a805-a3377dd45f14-kube-api-access-m46p9\") pod \"kuadrant-operator-catalog-bsqqk\" (UID: \"756f6c87-a8ca-4cff-a805-a3377dd45f14\") " pod="kuadrant-system/kuadrant-operator-catalog-bsqqk"
Jun 02 04:51:34.733440 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:34.733413    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m46p9\" (UniqueName: \"kubernetes.io/projected/756f6c87-a8ca-4cff-a805-a3377dd45f14-kube-api-access-m46p9\") pod \"kuadrant-operator-catalog-bsqqk\" (UID: \"756f6c87-a8ca-4cff-a805-a3377dd45f14\") " pod="kuadrant-system/kuadrant-operator-catalog-bsqqk"
Jun 02 04:51:34.743619 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:34.743596    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m46p9\" (UniqueName: \"kubernetes.io/projected/756f6c87-a8ca-4cff-a805-a3377dd45f14-kube-api-access-m46p9\") pod \"kuadrant-operator-catalog-bsqqk\" (UID: \"756f6c87-a8ca-4cff-a805-a3377dd45f14\") " pod="kuadrant-system/kuadrant-operator-catalog-bsqqk"
Jun 02 04:51:34.870314 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:34.870246    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bsqqk"
Jun 02 04:51:34.900874 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:34.900845    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xl4db" event={"ID":"a1a6a35f-b865-4101-b6df-00ec03f01a0a","Type":"ContainerStarted","Data":"247e1b2911007af6064488e23def890d7a1d21bfd4fd8c84f8a1ae39824c35fd"}
Jun 02 04:51:35.000429 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:35.000390    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bsqqk"]
Jun 02 04:51:35.036986 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:51:35.036955    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod756f6c87_a8ca_4cff_a805_a3377dd45f14.slice/crio-1c0bd2b130b3f458ba7063e8bf26d1e342f7ea3fc2722cf85fbd26afe41b1dc6 WatchSource:0}: Error finding container 1c0bd2b130b3f458ba7063e8bf26d1e342f7ea3fc2722cf85fbd26afe41b1dc6: Status 404 returned error can't find the container with id 1c0bd2b130b3f458ba7063e8bf26d1e342f7ea3fc2722cf85fbd26afe41b1dc6
Jun 02 04:51:35.905441 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:35.905408    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bsqqk" event={"ID":"756f6c87-a8ca-4cff-a805-a3377dd45f14","Type":"ContainerStarted","Data":"1c0bd2b130b3f458ba7063e8bf26d1e342f7ea3fc2722cf85fbd26afe41b1dc6"}
Jun 02 04:51:36.909717 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:36.909676    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bsqqk" event={"ID":"756f6c87-a8ca-4cff-a805-a3377dd45f14","Type":"ContainerStarted","Data":"e5dc11e6e78121de35f42de0b24c8181f551f033703ebfdc2a36a7d4e53fba16"}
Jun 02 04:51:36.910994 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:36.910968    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xl4db" event={"ID":"a1a6a35f-b865-4101-b6df-00ec03f01a0a","Type":"ContainerStarted","Data":"97bd831b89a192ad00405e0fcf5121045eb284082638cad28cd86a8611a804c3"}
Jun 02 04:51:36.911101 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:36.911017    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-xl4db" podUID="a1a6a35f-b865-4101-b6df-00ec03f01a0a" containerName="registry-server" containerID="cri-o://97bd831b89a192ad00405e0fcf5121045eb284082638cad28cd86a8611a804c3" gracePeriod=2
Jun 02 04:51:36.929050 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:36.929011    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-bsqqk" podStartSLOduration=1.6042658859999999 podStartE2EDuration="2.928998557s" podCreationTimestamp="2026-06-02 04:51:34 +0000 UTC" firstStartedPulling="2026-06-02 04:51:35.038363858 +0000 UTC m=+542.079294808" lastFinishedPulling="2026-06-02 04:51:36.363096529 +0000 UTC m=+543.404027479" observedRunningTime="2026-06-02 04:51:36.928102374 +0000 UTC m=+543.969033347" watchObservedRunningTime="2026-06-02 04:51:36.928998557 +0000 UTC m=+543.969929529"
Jun 02 04:51:36.945655 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:36.945616    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xl4db" podStartSLOduration=2.012941022 podStartE2EDuration="3.945603428s" podCreationTimestamp="2026-06-02 04:51:33 +0000 UTC" firstStartedPulling="2026-06-02 04:51:34.429959456 +0000 UTC m=+541.470890406" lastFinishedPulling="2026-06-02 04:51:36.362621843 +0000 UTC m=+543.403552812" observedRunningTime="2026-06-02 04:51:36.944285896 +0000 UTC m=+543.985216874" watchObservedRunningTime="2026-06-02 04:51:36.945603428 +0000 UTC m=+543.986534401"
Jun 02 04:51:37.150934 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:37.150907    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xl4db"
Jun 02 04:51:37.257394 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:37.257372    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnwg6\" (UniqueName: \"kubernetes.io/projected/a1a6a35f-b865-4101-b6df-00ec03f01a0a-kube-api-access-hnwg6\") pod \"a1a6a35f-b865-4101-b6df-00ec03f01a0a\" (UID: \"a1a6a35f-b865-4101-b6df-00ec03f01a0a\") "
Jun 02 04:51:37.259392 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:37.259368    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1a6a35f-b865-4101-b6df-00ec03f01a0a-kube-api-access-hnwg6" (OuterVolumeSpecName: "kube-api-access-hnwg6") pod "a1a6a35f-b865-4101-b6df-00ec03f01a0a" (UID: "a1a6a35f-b865-4101-b6df-00ec03f01a0a"). InnerVolumeSpecName "kube-api-access-hnwg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 04:51:37.357757 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:37.357729    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnwg6\" (UniqueName: \"kubernetes.io/projected/a1a6a35f-b865-4101-b6df-00ec03f01a0a-kube-api-access-hnwg6\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:51:37.915265 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:37.915237    2577 generic.go:358] "Generic (PLEG): container finished" podID="a1a6a35f-b865-4101-b6df-00ec03f01a0a" containerID="97bd831b89a192ad00405e0fcf5121045eb284082638cad28cd86a8611a804c3" exitCode=0
Jun 02 04:51:37.915642 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:37.915289    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xl4db"
Jun 02 04:51:37.915642 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:37.915323    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xl4db" event={"ID":"a1a6a35f-b865-4101-b6df-00ec03f01a0a","Type":"ContainerDied","Data":"97bd831b89a192ad00405e0fcf5121045eb284082638cad28cd86a8611a804c3"}
Jun 02 04:51:37.915642 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:37.915357    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xl4db" event={"ID":"a1a6a35f-b865-4101-b6df-00ec03f01a0a","Type":"ContainerDied","Data":"247e1b2911007af6064488e23def890d7a1d21bfd4fd8c84f8a1ae39824c35fd"}
Jun 02 04:51:37.915642 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:37.915372    2577 scope.go:117] "RemoveContainer" containerID="97bd831b89a192ad00405e0fcf5121045eb284082638cad28cd86a8611a804c3"
Jun 02 04:51:37.923534 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:37.923519    2577 scope.go:117] "RemoveContainer" containerID="97bd831b89a192ad00405e0fcf5121045eb284082638cad28cd86a8611a804c3"
Jun 02 04:51:37.923763 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:51:37.923746    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97bd831b89a192ad00405e0fcf5121045eb284082638cad28cd86a8611a804c3\": container with ID starting with 97bd831b89a192ad00405e0fcf5121045eb284082638cad28cd86a8611a804c3 not found: ID does not exist" containerID="97bd831b89a192ad00405e0fcf5121045eb284082638cad28cd86a8611a804c3"
Jun 02 04:51:37.923850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:37.923774    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97bd831b89a192ad00405e0fcf5121045eb284082638cad28cd86a8611a804c3"} err="failed to get container status \"97bd831b89a192ad00405e0fcf5121045eb284082638cad28cd86a8611a804c3\": rpc error: code = NotFound desc = could not find container \"97bd831b89a192ad00405e0fcf5121045eb284082638cad28cd86a8611a804c3\": container with ID starting with 97bd831b89a192ad00405e0fcf5121045eb284082638cad28cd86a8611a804c3 not found: ID does not exist"
Jun 02 04:51:37.931489 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:37.931464    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xl4db"]
Jun 02 04:51:37.934970 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:37.934949    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xl4db"]
Jun 02 04:51:39.444776 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:39.444739    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1a6a35f-b865-4101-b6df-00ec03f01a0a" path="/var/lib/kubelet/pods/a1a6a35f-b865-4101-b6df-00ec03f01a0a/volumes"
Jun 02 04:51:44.871300 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:44.871273    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-bsqqk"
Jun 02 04:51:44.871724 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:44.871495    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-bsqqk"
Jun 02 04:51:44.893273 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:44.893251    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-bsqqk"
Jun 02 04:51:44.959587 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:51:44.959561    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-bsqqk"
Jun 02 04:52:06.748138 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.748105    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-69b7b4b6dc-r6vxr"]
Jun 02 04:52:06.748579 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.748388    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1a6a35f-b865-4101-b6df-00ec03f01a0a" containerName="registry-server"
Jun 02 04:52:06.748579 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.748404    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1a6a35f-b865-4101-b6df-00ec03f01a0a" containerName="registry-server"
Jun 02 04:52:06.748579 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.748459    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="a1a6a35f-b865-4101-b6df-00ec03f01a0a" containerName="registry-server"
Jun 02 04:52:06.752976 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.752955    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.756240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.756214    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vhwvf\""
Jun 02 04:52:06.756240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.756233    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jun 02 04:52:06.757164 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.757143    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jun 02 04:52:06.757164 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.757158    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jun 02 04:52:06.757355 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.757253    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jun 02 04:52:06.757355 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.757349    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jun 02 04:52:06.758613 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.758579    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jun 02 04:52:06.767261 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.767236    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jun 02 04:52:06.767723 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.767701    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69b7b4b6dc-r6vxr"]
Jun 02 04:52:06.768073 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.768057    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jun 02 04:52:06.860209 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.860182    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ef79026-ad8d-49db-a5e0-d075fe7af24f-console-serving-cert\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.860308 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.860214    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ef79026-ad8d-49db-a5e0-d075fe7af24f-console-config\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.860308 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.860231    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ef79026-ad8d-49db-a5e0-d075fe7af24f-oauth-serving-cert\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.860308 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.860259    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ef79026-ad8d-49db-a5e0-d075fe7af24f-console-oauth-config\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.860421 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.860324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ef79026-ad8d-49db-a5e0-d075fe7af24f-trusted-ca-bundle\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.860421 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.860352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ef79026-ad8d-49db-a5e0-d075fe7af24f-service-ca\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.860421 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.860370    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x9nf\" (UniqueName: \"kubernetes.io/projected/0ef79026-ad8d-49db-a5e0-d075fe7af24f-kube-api-access-2x9nf\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.960612 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.960591    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ef79026-ad8d-49db-a5e0-d075fe7af24f-console-serving-cert\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.960735 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.960623    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ef79026-ad8d-49db-a5e0-d075fe7af24f-console-config\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.960735 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.960644    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ef79026-ad8d-49db-a5e0-d075fe7af24f-oauth-serving-cert\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.960735 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.960679    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ef79026-ad8d-49db-a5e0-d075fe7af24f-console-oauth-config\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.960907 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.960859    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ef79026-ad8d-49db-a5e0-d075fe7af24f-trusted-ca-bundle\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.960907 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.960895    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ef79026-ad8d-49db-a5e0-d075fe7af24f-service-ca\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.961013 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.960924    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2x9nf\" (UniqueName: \"kubernetes.io/projected/0ef79026-ad8d-49db-a5e0-d075fe7af24f-kube-api-access-2x9nf\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.961528 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.961501    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ef79026-ad8d-49db-a5e0-d075fe7af24f-console-config\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.961628 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.961534    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ef79026-ad8d-49db-a5e0-d075fe7af24f-service-ca\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.961628 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.961563    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ef79026-ad8d-49db-a5e0-d075fe7af24f-oauth-serving-cert\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.961732 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.961663    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ef79026-ad8d-49db-a5e0-d075fe7af24f-trusted-ca-bundle\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.963173 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.963152    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ef79026-ad8d-49db-a5e0-d075fe7af24f-console-oauth-config\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.963260 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.963178    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ef79026-ad8d-49db-a5e0-d075fe7af24f-console-serving-cert\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:06.969388 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:06.969367    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x9nf\" (UniqueName: \"kubernetes.io/projected/0ef79026-ad8d-49db-a5e0-d075fe7af24f-kube-api-access-2x9nf\") pod \"console-69b7b4b6dc-r6vxr\" (UID: \"0ef79026-ad8d-49db-a5e0-d075fe7af24f\") " pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:07.064734 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:07.064683    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:07.193093 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:07.193037    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69b7b4b6dc-r6vxr"]
Jun 02 04:52:07.195417 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:52:07.195389    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ef79026_ad8d_49db_a5e0_d075fe7af24f.slice/crio-c87d9b77d8254ada38fe83be2b56b16300954e0fe963878d62ff31caeddeb04d WatchSource:0}: Error finding container c87d9b77d8254ada38fe83be2b56b16300954e0fe963878d62ff31caeddeb04d: Status 404 returned error can't find the container with id c87d9b77d8254ada38fe83be2b56b16300954e0fe963878d62ff31caeddeb04d
Jun 02 04:52:08.013434 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:08.013392    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69b7b4b6dc-r6vxr" event={"ID":"0ef79026-ad8d-49db-a5e0-d075fe7af24f","Type":"ContainerStarted","Data":"d238f33c3cb244b9d7cf32b2b896875cf67cc6ec5c894cd5d658baef6a313012"}
Jun 02 04:52:08.013434 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:08.013429    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69b7b4b6dc-r6vxr" event={"ID":"0ef79026-ad8d-49db-a5e0-d075fe7af24f","Type":"ContainerStarted","Data":"c87d9b77d8254ada38fe83be2b56b16300954e0fe963878d62ff31caeddeb04d"}
Jun 02 04:52:08.033961 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:08.033922    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-69b7b4b6dc-r6vxr" podStartSLOduration=2.033909346 podStartE2EDuration="2.033909346s" podCreationTimestamp="2026-06-02 04:52:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 04:52:08.031665551 +0000 UTC m=+575.072596523" watchObservedRunningTime="2026-06-02 04:52:08.033909346 +0000 UTC m=+575.074840318"
Jun 02 04:52:15.800357 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:15.800322    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"]
Jun 02 04:52:15.806426 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:15.806405    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"
Jun 02 04:52:15.809124 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:15.809106    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7jnt5\""
Jun 02 04:52:15.814427 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:15.814406    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"]
Jun 02 04:52:15.930370 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:15.930345    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" (UID: \"7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"
Jun 02 04:52:15.930465 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:15.930384    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9n5w\" (UniqueName: \"kubernetes.io/projected/7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca-kube-api-access-t9n5w\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" (UID: \"7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"
Jun 02 04:52:16.031341 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:16.031315    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" (UID: \"7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"
Jun 02 04:52:16.031428 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:16.031348    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t9n5w\" (UniqueName: \"kubernetes.io/projected/7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca-kube-api-access-t9n5w\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" (UID: \"7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"
Jun 02 04:52:16.031723 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:16.031705    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" (UID: \"7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"
Jun 02 04:52:16.039854 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:16.039827    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9n5w\" (UniqueName: \"kubernetes.io/projected/7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca-kube-api-access-t9n5w\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" (UID: \"7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"
Jun 02 04:52:16.117939 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:16.117876    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"
Jun 02 04:52:16.248450 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:16.248421    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"]
Jun 02 04:52:16.251057 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:52:16.251033    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e8f8f37_43aa_4cf6_9e7d_8e97f48a1fca.slice/crio-33972ece6c830b3e8aabc5adcfc56c8735d53e64305a8764f4eae289723e3d09 WatchSource:0}: Error finding container 33972ece6c830b3e8aabc5adcfc56c8735d53e64305a8764f4eae289723e3d09: Status 404 returned error can't find the container with id 33972ece6c830b3e8aabc5adcfc56c8735d53e64305a8764f4eae289723e3d09
Jun 02 04:52:17.050273 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:17.050173    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" event={"ID":"7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca","Type":"ContainerStarted","Data":"33972ece6c830b3e8aabc5adcfc56c8735d53e64305a8764f4eae289723e3d09"}
Jun 02 04:52:17.065824 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:17.065779    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:17.066020 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:17.065839    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:17.071516 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:17.071494    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:18.058454 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:18.058427    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69b7b4b6dc-r6vxr"
Jun 02 04:52:19.021918 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:19.021884    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-twmzt"]
Jun 02 04:52:19.025469 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:19.025452    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-twmzt"
Jun 02 04:52:19.028040 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:19.028018    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-zlsqf\""
Jun 02 04:52:19.028040 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:19.028037    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jun 02 04:52:19.036248 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:19.036225    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-twmzt"]
Jun 02 04:52:19.157252 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:19.156462    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58vbm\" (UniqueName: \"kubernetes.io/projected/df47cc68-409f-453a-b88a-cb8b6b3929b3-kube-api-access-58vbm\") pod \"dns-operator-controller-manager-648d5c98bc-twmzt\" (UID: \"df47cc68-409f-453a-b88a-cb8b6b3929b3\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-twmzt"
Jun 02 04:52:19.257569 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:19.257534    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-58vbm\" (UniqueName: \"kubernetes.io/projected/df47cc68-409f-453a-b88a-cb8b6b3929b3-kube-api-access-58vbm\") pod \"dns-operator-controller-manager-648d5c98bc-twmzt\" (UID: \"df47cc68-409f-453a-b88a-cb8b6b3929b3\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-twmzt"
Jun 02 04:52:19.266867 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:19.266840    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58vbm\" (UniqueName: \"kubernetes.io/projected/df47cc68-409f-453a-b88a-cb8b6b3929b3-kube-api-access-58vbm\") pod \"dns-operator-controller-manager-648d5c98bc-twmzt\" (UID: \"df47cc68-409f-453a-b88a-cb8b6b3929b3\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-twmzt"
Jun 02 04:52:19.336022 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:19.335952    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-twmzt"
Jun 02 04:52:20.560133 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:20.560111    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-twmzt"]
Jun 02 04:52:20.563190 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:52:20.563160    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf47cc68_409f_453a_b88a_cb8b6b3929b3.slice/crio-15f8909f93e5312ccb02b91aecae7129febd2f809372f4bbd6a0bc9b7da5dbaf WatchSource:0}: Error finding container 15f8909f93e5312ccb02b91aecae7129febd2f809372f4bbd6a0bc9b7da5dbaf: Status 404 returned error can't find the container with id 15f8909f93e5312ccb02b91aecae7129febd2f809372f4bbd6a0bc9b7da5dbaf
Jun 02 04:52:21.065694 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:21.065647    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-twmzt" event={"ID":"df47cc68-409f-453a-b88a-cb8b6b3929b3","Type":"ContainerStarted","Data":"15f8909f93e5312ccb02b91aecae7129febd2f809372f4bbd6a0bc9b7da5dbaf"}
Jun 02 04:52:21.067060 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:21.067035    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" event={"ID":"7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca","Type":"ContainerStarted","Data":"a159940dad51778aff13e84da6a9e743789b45c151ff363b765961c1f453a005"}
Jun 02 04:52:21.067174 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:21.067150    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"
Jun 02 04:52:21.089826 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:21.089771    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" podStartSLOduration=1.847041048 podStartE2EDuration="6.089759121s" podCreationTimestamp="2026-06-02 04:52:15 +0000 UTC" firstStartedPulling="2026-06-02 04:52:16.253792495 +0000 UTC m=+583.294723449" lastFinishedPulling="2026-06-02 04:52:20.496510557 +0000 UTC m=+587.537441522" observedRunningTime="2026-06-02 04:52:21.087606145 +0000 UTC m=+588.128537117" watchObservedRunningTime="2026-06-02 04:52:21.089759121 +0000 UTC m=+588.130690093"
Jun 02 04:52:22.078102 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:22.078073    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts"]
Jun 02 04:52:22.081337 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:22.081319    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts"
Jun 02 04:52:22.084101 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:22.084080    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hmjfq\""
Jun 02 04:52:22.090215 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:22.090196    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts"]
Jun 02 04:52:22.182295 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:22.182262    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mprx\" (UniqueName: \"kubernetes.io/projected/5583e9af-bb38-415d-8119-63d6ebabe184-kube-api-access-8mprx\") pod \"limitador-operator-controller-manager-85c4996f8c-t44ts\" (UID: \"5583e9af-bb38-415d-8119-63d6ebabe184\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts"
Jun 02 04:52:22.283410 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:22.283377    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8mprx\" (UniqueName: \"kubernetes.io/projected/5583e9af-bb38-415d-8119-63d6ebabe184-kube-api-access-8mprx\") pod \"limitador-operator-controller-manager-85c4996f8c-t44ts\" (UID: \"5583e9af-bb38-415d-8119-63d6ebabe184\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts"
Jun 02 04:52:22.292508 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:22.292482    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mprx\" (UniqueName: \"kubernetes.io/projected/5583e9af-bb38-415d-8119-63d6ebabe184-kube-api-access-8mprx\") pod \"limitador-operator-controller-manager-85c4996f8c-t44ts\" (UID: \"5583e9af-bb38-415d-8119-63d6ebabe184\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts"
Jun 02 04:52:22.394389 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:22.394313    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts"
Jun 02 04:52:22.536498 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:22.536431    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts"]
Jun 02 04:52:22.538712 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:52:22.538688    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5583e9af_bb38_415d_8119_63d6ebabe184.slice/crio-63d989329fd2827d6d5158d6cab18ad0030cd6956350d693ac8c302b53ca05cb WatchSource:0}: Error finding container 63d989329fd2827d6d5158d6cab18ad0030cd6956350d693ac8c302b53ca05cb: Status 404 returned error can't find the container with id 63d989329fd2827d6d5158d6cab18ad0030cd6956350d693ac8c302b53ca05cb
Jun 02 04:52:23.077651 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:23.077618    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts" event={"ID":"5583e9af-bb38-415d-8119-63d6ebabe184","Type":"ContainerStarted","Data":"63d989329fd2827d6d5158d6cab18ad0030cd6956350d693ac8c302b53ca05cb"}
Jun 02 04:52:23.079192 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:23.079166    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-twmzt" event={"ID":"df47cc68-409f-453a-b88a-cb8b6b3929b3","Type":"ContainerStarted","Data":"15d29d8eb87cd2bae0bc5d7cd6195c1a8ab7d780b2badd1c4a3f7e43019d852f"}
Jun 02 04:52:23.079534 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:23.079355    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-twmzt"
Jun 02 04:52:23.101508 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:23.101466    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-twmzt" podStartSLOduration=2.2205564349999998 podStartE2EDuration="4.101453223s" podCreationTimestamp="2026-06-02 04:52:19 +0000 UTC" firstStartedPulling="2026-06-02 04:52:20.565588298 +0000 UTC m=+587.606519247" lastFinishedPulling="2026-06-02 04:52:22.446485081 +0000 UTC m=+589.487416035" observedRunningTime="2026-06-02 04:52:23.100213108 +0000 UTC m=+590.141144081" watchObservedRunningTime="2026-06-02 04:52:23.101453223 +0000 UTC m=+590.142384195"
Jun 02 04:52:25.087693 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:25.087656    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts" event={"ID":"5583e9af-bb38-415d-8119-63d6ebabe184","Type":"ContainerStarted","Data":"1ae9eff80df007cdcb90c08a4446073aa745ff4251c08cb7be1e3e376fe68211"}
Jun 02 04:52:25.088059 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:25.087741    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts"
Jun 02 04:52:25.104381 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:25.104339    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts" podStartSLOduration=1.00290324 podStartE2EDuration="3.104325405s" podCreationTimestamp="2026-06-02 04:52:22 +0000 UTC" firstStartedPulling="2026-06-02 04:52:22.540468586 +0000 UTC m=+589.581399536" lastFinishedPulling="2026-06-02 04:52:24.641890751 +0000 UTC m=+591.682821701" observedRunningTime="2026-06-02 04:52:25.104174685 +0000 UTC m=+592.145105658" watchObservedRunningTime="2026-06-02 04:52:25.104325405 +0000 UTC m=+592.145256377"
Jun 02 04:52:32.074217 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:32.074183    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"
Jun 02 04:52:33.375894 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.375867    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 04:52:33.376280 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.376074    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 04:52:33.793922 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.793895    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"]
Jun 02 04:52:33.794115 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.794094    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" containerName="manager" containerID="cri-o://a159940dad51778aff13e84da6a9e743789b45c151ff363b765961c1f453a005" gracePeriod=2
Jun 02 04:52:33.796649 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.796462    2577 status_manager.go:895] "Failed to get status for pod" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:33.797573 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.797548    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"]
Jun 02 04:52:33.809544 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.809516    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts"]
Jun 02 04:52:33.809842 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.809792    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts" podUID="5583e9af-bb38-415d-8119-63d6ebabe184" containerName="manager" containerID="cri-o://1ae9eff80df007cdcb90c08a4446073aa745ff4251c08cb7be1e3e376fe68211" gracePeriod=2
Jun 02 04:52:33.812062 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.812041    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts"
Jun 02 04:52:33.816695 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.816552    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"]
Jun 02 04:52:33.817101 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.817082    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" containerName="manager"
Jun 02 04:52:33.817199 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.817103    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" containerName="manager"
Jun 02 04:52:33.817199 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.817188    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" containerName="manager"
Jun 02 04:52:33.820137 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.820117    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts"]
Jun 02 04:52:33.820229 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.820217    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"
Jun 02 04:52:33.827102 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.827071    2577 status_manager.go:895] "Failed to get status for pod" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:33.830955 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.830932    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-hrbx7"]
Jun 02 04:52:33.831405 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.831386    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5583e9af-bb38-415d-8119-63d6ebabe184" containerName="manager"
Jun 02 04:52:33.831405 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.831406    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5583e9af-bb38-415d-8119-63d6ebabe184" containerName="manager"
Jun 02 04:52:33.831556 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.831503    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="5583e9af-bb38-415d-8119-63d6ebabe184" containerName="manager"
Jun 02 04:52:33.834457 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.834437    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"]
Jun 02 04:52:33.834560 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.834545    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-hrbx7"
Jun 02 04:52:33.855694 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.855670    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-hrbx7"]
Jun 02 04:52:33.872194 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.872160    2577 status_manager.go:895] "Failed to get status for pod" podUID="5583e9af-bb38-415d-8119-63d6ebabe184" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts" err="pods \"limitador-operator-controller-manager-85c4996f8c-t44ts\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:33.877351 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.877330    2577 status_manager.go:895] "Failed to get status for pod" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:33.907161 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.907137    2577 status_manager.go:895] "Failed to get status for pod" podUID="5583e9af-bb38-415d-8119-63d6ebabe184" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts" err="pods \"limitador-operator-controller-manager-85c4996f8c-t44ts\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:33.909754 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.909729    2577 status_manager.go:895] "Failed to get status for pod" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:33.973399 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.973365    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsrxx\" (UniqueName: \"kubernetes.io/projected/f7cc85f3-afe1-4cff-aaa7-55c01da30af2-kube-api-access-bsrxx\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-6cdwq\" (UID: \"f7cc85f3-afe1-4cff-aaa7-55c01da30af2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"
Jun 02 04:52:33.974177 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.973539    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg9nf\" (UniqueName: \"kubernetes.io/projected/82ae05cd-8b0d-44f0-8b44-cbcaf39770ca-kube-api-access-gg9nf\") pod \"limitador-operator-controller-manager-85c4996f8c-hrbx7\" (UID: \"82ae05cd-8b0d-44f0-8b44-cbcaf39770ca\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-hrbx7"
Jun 02 04:52:33.974177 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:33.973586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7cc85f3-afe1-4cff-aaa7-55c01da30af2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-6cdwq\" (UID: \"f7cc85f3-afe1-4cff-aaa7-55c01da30af2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"
Jun 02 04:52:34.041967 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.041946    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"
Jun 02 04:52:34.044495 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.044432    2577 status_manager.go:895] "Failed to get status for pod" podUID="5583e9af-bb38-415d-8119-63d6ebabe184" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts" err="pods \"limitador-operator-controller-manager-85c4996f8c-t44ts\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:34.045072 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.045057    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts"
Jun 02 04:52:34.046424 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.046404    2577 status_manager.go:895] "Failed to get status for pod" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:34.048425 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.048406    2577 status_manager.go:895] "Failed to get status for pod" podUID="5583e9af-bb38-415d-8119-63d6ebabe184" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts" err="pods \"limitador-operator-controller-manager-85c4996f8c-t44ts\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:34.050395 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.050377    2577 status_manager.go:895] "Failed to get status for pod" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:34.074804 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.074780    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gg9nf\" (UniqueName: \"kubernetes.io/projected/82ae05cd-8b0d-44f0-8b44-cbcaf39770ca-kube-api-access-gg9nf\") pod \"limitador-operator-controller-manager-85c4996f8c-hrbx7\" (UID: \"82ae05cd-8b0d-44f0-8b44-cbcaf39770ca\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-hrbx7"
Jun 02 04:52:34.074918 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.074836    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7cc85f3-afe1-4cff-aaa7-55c01da30af2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-6cdwq\" (UID: \"f7cc85f3-afe1-4cff-aaa7-55c01da30af2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"
Jun 02 04:52:34.074918 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.074880    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bsrxx\" (UniqueName: \"kubernetes.io/projected/f7cc85f3-afe1-4cff-aaa7-55c01da30af2-kube-api-access-bsrxx\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-6cdwq\" (UID: \"f7cc85f3-afe1-4cff-aaa7-55c01da30af2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"
Jun 02 04:52:34.075192 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.075170    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7cc85f3-afe1-4cff-aaa7-55c01da30af2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-6cdwq\" (UID: \"f7cc85f3-afe1-4cff-aaa7-55c01da30af2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"
Jun 02 04:52:34.083466 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.083436    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsrxx\" (UniqueName: \"kubernetes.io/projected/f7cc85f3-afe1-4cff-aaa7-55c01da30af2-kube-api-access-bsrxx\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-6cdwq\" (UID: \"f7cc85f3-afe1-4cff-aaa7-55c01da30af2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"
Jun 02 04:52:34.083586 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.083565    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg9nf\" (UniqueName: \"kubernetes.io/projected/82ae05cd-8b0d-44f0-8b44-cbcaf39770ca-kube-api-access-gg9nf\") pod \"limitador-operator-controller-manager-85c4996f8c-hrbx7\" (UID: \"82ae05cd-8b0d-44f0-8b44-cbcaf39770ca\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-hrbx7"
Jun 02 04:52:34.084886 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.084869    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-twmzt"
Jun 02 04:52:34.087225 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.087204    2577 status_manager.go:895] "Failed to get status for pod" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:34.089343 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.089321    2577 status_manager.go:895] "Failed to get status for pod" podUID="5583e9af-bb38-415d-8119-63d6ebabe184" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts" err="pods \"limitador-operator-controller-manager-85c4996f8c-t44ts\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:34.117232 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.117214    2577 generic.go:358] "Generic (PLEG): container finished" podID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" containerID="a159940dad51778aff13e84da6a9e743789b45c151ff363b765961c1f453a005" exitCode=0
Jun 02 04:52:34.117315 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.117249    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw"
Jun 02 04:52:34.117315 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.117306    2577 scope.go:117] "RemoveContainer" containerID="a159940dad51778aff13e84da6a9e743789b45c151ff363b765961c1f453a005"
Jun 02 04:52:34.118737 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.118703    2577 generic.go:358] "Generic (PLEG): container finished" podID="5583e9af-bb38-415d-8119-63d6ebabe184" containerID="1ae9eff80df007cdcb90c08a4446073aa745ff4251c08cb7be1e3e376fe68211" exitCode=0
Jun 02 04:52:34.118884 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.118870    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts"
Jun 02 04:52:34.119874 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.119847    2577 status_manager.go:895] "Failed to get status for pod" podUID="5583e9af-bb38-415d-8119-63d6ebabe184" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts" err="pods \"limitador-operator-controller-manager-85c4996f8c-t44ts\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:34.122053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.122020    2577 status_manager.go:895] "Failed to get status for pod" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:34.124209 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.124185    2577 status_manager.go:895] "Failed to get status for pod" podUID="5583e9af-bb38-415d-8119-63d6ebabe184" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts" err="pods \"limitador-operator-controller-manager-85c4996f8c-t44ts\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:34.126055 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.125977    2577 status_manager.go:895] "Failed to get status for pod" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:34.126128 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.126113    2577 scope.go:117] "RemoveContainer" containerID="a159940dad51778aff13e84da6a9e743789b45c151ff363b765961c1f453a005"
Jun 02 04:52:34.126398 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:52:34.126377    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a159940dad51778aff13e84da6a9e743789b45c151ff363b765961c1f453a005\": container with ID starting with a159940dad51778aff13e84da6a9e743789b45c151ff363b765961c1f453a005 not found: ID does not exist" containerID="a159940dad51778aff13e84da6a9e743789b45c151ff363b765961c1f453a005"
Jun 02 04:52:34.126451 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.126408    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a159940dad51778aff13e84da6a9e743789b45c151ff363b765961c1f453a005"} err="failed to get container status \"a159940dad51778aff13e84da6a9e743789b45c151ff363b765961c1f453a005\": rpc error: code = NotFound desc = could not find container \"a159940dad51778aff13e84da6a9e743789b45c151ff363b765961c1f453a005\": container with ID starting with a159940dad51778aff13e84da6a9e743789b45c151ff363b765961c1f453a005 not found: ID does not exist"
Jun 02 04:52:34.126451 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.126430    2577 scope.go:117] "RemoveContainer" containerID="1ae9eff80df007cdcb90c08a4446073aa745ff4251c08cb7be1e3e376fe68211"
Jun 02 04:52:34.134342 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.134320    2577 scope.go:117] "RemoveContainer" containerID="1ae9eff80df007cdcb90c08a4446073aa745ff4251c08cb7be1e3e376fe68211"
Jun 02 04:52:34.134600 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:52:34.134584    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ae9eff80df007cdcb90c08a4446073aa745ff4251c08cb7be1e3e376fe68211\": container with ID starting with 1ae9eff80df007cdcb90c08a4446073aa745ff4251c08cb7be1e3e376fe68211 not found: ID does not exist" containerID="1ae9eff80df007cdcb90c08a4446073aa745ff4251c08cb7be1e3e376fe68211"
Jun 02 04:52:34.134681 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.134606    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ae9eff80df007cdcb90c08a4446073aa745ff4251c08cb7be1e3e376fe68211"} err="failed to get container status \"1ae9eff80df007cdcb90c08a4446073aa745ff4251c08cb7be1e3e376fe68211\": rpc error: code = NotFound desc = could not find container \"1ae9eff80df007cdcb90c08a4446073aa745ff4251c08cb7be1e3e376fe68211\": container with ID starting with 1ae9eff80df007cdcb90c08a4446073aa745ff4251c08cb7be1e3e376fe68211 not found: ID does not exist"
Jun 02 04:52:34.175781 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.175761    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9n5w\" (UniqueName: \"kubernetes.io/projected/7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca-kube-api-access-t9n5w\") pod \"7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca\" (UID: \"7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca\") "
Jun 02 04:52:34.175895 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.175827    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca-extensions-socket-volume\") pod \"7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca\" (UID: \"7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca\") "
Jun 02 04:52:34.175943 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.175901    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mprx\" (UniqueName: \"kubernetes.io/projected/5583e9af-bb38-415d-8119-63d6ebabe184-kube-api-access-8mprx\") pod \"5583e9af-bb38-415d-8119-63d6ebabe184\" (UID: \"5583e9af-bb38-415d-8119-63d6ebabe184\") "
Jun 02 04:52:34.176175 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.176157    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" (UID: "7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 04:52:34.177764 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.177743    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca-kube-api-access-t9n5w" (OuterVolumeSpecName: "kube-api-access-t9n5w") pod "7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" (UID: "7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca"). InnerVolumeSpecName "kube-api-access-t9n5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 04:52:34.177852 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.177769    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5583e9af-bb38-415d-8119-63d6ebabe184-kube-api-access-8mprx" (OuterVolumeSpecName: "kube-api-access-8mprx") pod "5583e9af-bb38-415d-8119-63d6ebabe184" (UID: "5583e9af-bb38-415d-8119-63d6ebabe184"). InnerVolumeSpecName "kube-api-access-8mprx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 04:52:34.241638 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.241614    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"
Jun 02 04:52:34.250271 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.250256    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-hrbx7"
Jun 02 04:52:34.278218 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.278184    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca-extensions-socket-volume\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:52:34.278350 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.278221    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mprx\" (UniqueName: \"kubernetes.io/projected/5583e9af-bb38-415d-8119-63d6ebabe184-kube-api-access-8mprx\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:52:34.278350 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.278240    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9n5w\" (UniqueName: \"kubernetes.io/projected/7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca-kube-api-access-t9n5w\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:52:34.381018 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.380790    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"]
Jun 02 04:52:34.385217 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:52:34.385174    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7cc85f3_afe1_4cff_aaa7_55c01da30af2.slice/crio-2a08a0dbd202d2fc158ac3a2838a68aa7c8d228ac4674f81fc53947d7bc71bd9 WatchSource:0}: Error finding container 2a08a0dbd202d2fc158ac3a2838a68aa7c8d228ac4674f81fc53947d7bc71bd9: Status 404 returned error can't find the container with id 2a08a0dbd202d2fc158ac3a2838a68aa7c8d228ac4674f81fc53947d7bc71bd9
Jun 02 04:52:34.402674 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.402656    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-hrbx7"]
Jun 02 04:52:34.404219 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:52:34.404199    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ae05cd_8b0d_44f0_8b44_cbcaf39770ca.slice/crio-58ab4e189a04261e1c7b9b3e4819a86fe3f2f6f6319c0805df47028885d9574f WatchSource:0}: Error finding container 58ab4e189a04261e1c7b9b3e4819a86fe3f2f6f6319c0805df47028885d9574f: Status 404 returned error can't find the container with id 58ab4e189a04261e1c7b9b3e4819a86fe3f2f6f6319c0805df47028885d9574f
Jun 02 04:52:34.430022 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.429998    2577 status_manager.go:895] "Failed to get status for pod" podUID="5583e9af-bb38-415d-8119-63d6ebabe184" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts" err="pods \"limitador-operator-controller-manager-85c4996f8c-t44ts\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:34.432137 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.432109    2577 status_manager.go:895] "Failed to get status for pod" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:34.438250 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.438226    2577 status_manager.go:895] "Failed to get status for pod" podUID="5583e9af-bb38-415d-8119-63d6ebabe184" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts" err="pods \"limitador-operator-controller-manager-85c4996f8c-t44ts\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:34.440028 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:34.439994    2577 status_manager.go:895] "Failed to get status for pod" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:35.124293 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:35.124206    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-hrbx7" event={"ID":"82ae05cd-8b0d-44f0-8b44-cbcaf39770ca","Type":"ContainerStarted","Data":"929291ec4c9d92f62bb71b2a283bcd8d07bec6d61f7d3eb2287e725181a12232"}
Jun 02 04:52:35.124293 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:35.124245    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-hrbx7" event={"ID":"82ae05cd-8b0d-44f0-8b44-cbcaf39770ca","Type":"ContainerStarted","Data":"58ab4e189a04261e1c7b9b3e4819a86fe3f2f6f6319c0805df47028885d9574f"}
Jun 02 04:52:35.124496 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:35.124441    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-hrbx7"
Jun 02 04:52:35.125692 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:35.125667    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq" event={"ID":"f7cc85f3-afe1-4cff-aaa7-55c01da30af2","Type":"ContainerStarted","Data":"3d73535e0f817434bcbe31869ecda0a820978a50654b7276443cb0cfd11ab5c5"}
Jun 02 04:52:35.125779 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:35.125699    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq" event={"ID":"f7cc85f3-afe1-4cff-aaa7-55c01da30af2","Type":"ContainerStarted","Data":"2a08a0dbd202d2fc158ac3a2838a68aa7c8d228ac4674f81fc53947d7bc71bd9"}
Jun 02 04:52:35.125849 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:35.125791    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"
Jun 02 04:52:35.159043 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:35.159003    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-hrbx7" podStartSLOduration=2.158991206 podStartE2EDuration="2.158991206s" podCreationTimestamp="2026-06-02 04:52:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 04:52:35.157203915 +0000 UTC m=+602.198134879" watchObservedRunningTime="2026-06-02 04:52:35.158991206 +0000 UTC m=+602.199922177"
Jun 02 04:52:35.159896 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:35.159869    2577 status_manager.go:895] "Failed to get status for pod" podUID="5583e9af-bb38-415d-8119-63d6ebabe184" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts" err="pods \"limitador-operator-controller-manager-85c4996f8c-t44ts\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:35.162306 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:35.162287    2577 status_manager.go:895] "Failed to get status for pod" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:35.223492 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:35.223464    2577 status_manager.go:895] "Failed to get status for pod" podUID="5583e9af-bb38-415d-8119-63d6ebabe184" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-t44ts" err="pods \"limitador-operator-controller-manager-85c4996f8c-t44ts\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:35.223775 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:35.223741    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq" podStartSLOduration=2.223728541 podStartE2EDuration="2.223728541s" podCreationTimestamp="2026-06-02 04:52:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 04:52:35.22080611 +0000 UTC m=+602.261737081" watchObservedRunningTime="2026-06-02 04:52:35.223728541 +0000 UTC m=+602.264659527"
Jun 02 04:52:35.225870 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:35.225849    2577 status_manager.go:895] "Failed to get status for pod" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-chcfw" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-chcfw\" is forbidden: User \"system:node:ip-10-0-135-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-23.ec2.internal' and this object"
Jun 02 04:52:35.445767 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:35.445700    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5583e9af-bb38-415d-8119-63d6ebabe184" path="/var/lib/kubelet/pods/5583e9af-bb38-415d-8119-63d6ebabe184/volumes"
Jun 02 04:52:35.446078 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:35.446057    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca" path="/var/lib/kubelet/pods/7e8f8f37-43aa-4cf6-9e7d-8e97f48a1fca/volumes"
Jun 02 04:52:46.132077 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:46.132050    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-hrbx7"
Jun 02 04:52:46.135892 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:46.132203    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"
Jun 02 04:52:52.242458 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:52.242425    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"]
Jun 02 04:52:52.242834 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:52.242656    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq" podUID="f7cc85f3-afe1-4cff-aaa7-55c01da30af2" containerName="manager" containerID="cri-o://3d73535e0f817434bcbe31869ecda0a820978a50654b7276443cb0cfd11ab5c5" gracePeriod=10
Jun 02 04:52:52.885309 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:52.885286    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"
Jun 02 04:52:52.919713 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:52.919688    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7cc85f3-afe1-4cff-aaa7-55c01da30af2-extensions-socket-volume\") pod \"f7cc85f3-afe1-4cff-aaa7-55c01da30af2\" (UID: \"f7cc85f3-afe1-4cff-aaa7-55c01da30af2\") "
Jun 02 04:52:52.919875 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:52.919743    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsrxx\" (UniqueName: \"kubernetes.io/projected/f7cc85f3-afe1-4cff-aaa7-55c01da30af2-kube-api-access-bsrxx\") pod \"f7cc85f3-afe1-4cff-aaa7-55c01da30af2\" (UID: \"f7cc85f3-afe1-4cff-aaa7-55c01da30af2\") "
Jun 02 04:52:52.920093 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:52.920068    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7cc85f3-afe1-4cff-aaa7-55c01da30af2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f7cc85f3-afe1-4cff-aaa7-55c01da30af2" (UID: "f7cc85f3-afe1-4cff-aaa7-55c01da30af2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jun 02 04:52:52.921757 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:52.921735    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7cc85f3-afe1-4cff-aaa7-55c01da30af2-kube-api-access-bsrxx" (OuterVolumeSpecName: "kube-api-access-bsrxx") pod "f7cc85f3-afe1-4cff-aaa7-55c01da30af2" (UID: "f7cc85f3-afe1-4cff-aaa7-55c01da30af2"). InnerVolumeSpecName "kube-api-access-bsrxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 04:52:53.020251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:53.020226    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7cc85f3-afe1-4cff-aaa7-55c01da30af2-extensions-socket-volume\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:52:53.020251 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:53.020250    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsrxx\" (UniqueName: \"kubernetes.io/projected/f7cc85f3-afe1-4cff-aaa7-55c01da30af2-kube-api-access-bsrxx\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:52:53.186583 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:53.186556    2577 generic.go:358] "Generic (PLEG): container finished" podID="f7cc85f3-afe1-4cff-aaa7-55c01da30af2" containerID="3d73535e0f817434bcbe31869ecda0a820978a50654b7276443cb0cfd11ab5c5" exitCode=0
Jun 02 04:52:53.186699 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:53.186616    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"
Jun 02 04:52:53.186699 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:53.186635    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq" event={"ID":"f7cc85f3-afe1-4cff-aaa7-55c01da30af2","Type":"ContainerDied","Data":"3d73535e0f817434bcbe31869ecda0a820978a50654b7276443cb0cfd11ab5c5"}
Jun 02 04:52:53.186699 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:53.186671    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq" event={"ID":"f7cc85f3-afe1-4cff-aaa7-55c01da30af2","Type":"ContainerDied","Data":"2a08a0dbd202d2fc158ac3a2838a68aa7c8d228ac4674f81fc53947d7bc71bd9"}
Jun 02 04:52:53.186699 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:53.186691    2577 scope.go:117] "RemoveContainer" containerID="3d73535e0f817434bcbe31869ecda0a820978a50654b7276443cb0cfd11ab5c5"
Jun 02 04:52:53.195148 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:53.194939    2577 scope.go:117] "RemoveContainer" containerID="3d73535e0f817434bcbe31869ecda0a820978a50654b7276443cb0cfd11ab5c5"
Jun 02 04:52:53.195221 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:52:53.195201    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d73535e0f817434bcbe31869ecda0a820978a50654b7276443cb0cfd11ab5c5\": container with ID starting with 3d73535e0f817434bcbe31869ecda0a820978a50654b7276443cb0cfd11ab5c5 not found: ID does not exist" containerID="3d73535e0f817434bcbe31869ecda0a820978a50654b7276443cb0cfd11ab5c5"
Jun 02 04:52:53.195262 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:53.195229    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d73535e0f817434bcbe31869ecda0a820978a50654b7276443cb0cfd11ab5c5"} err="failed to get container status \"3d73535e0f817434bcbe31869ecda0a820978a50654b7276443cb0cfd11ab5c5\": rpc error: code = NotFound desc = could not find container \"3d73535e0f817434bcbe31869ecda0a820978a50654b7276443cb0cfd11ab5c5\": container with ID starting with 3d73535e0f817434bcbe31869ecda0a820978a50654b7276443cb0cfd11ab5c5 not found: ID does not exist"
Jun 02 04:52:53.209066 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:53.209036    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"]
Jun 02 04:52:53.213320 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:53.213299    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-6cdwq"]
Jun 02 04:52:53.444204 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:52:53.444150    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7cc85f3-afe1-4cff-aaa7-55c01da30af2" path="/var/lib/kubelet/pods/f7cc85f3-afe1-4cff-aaa7-55c01da30af2/volumes"
Jun 02 04:53:08.598747 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.598715    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"]
Jun 02 04:53:08.599228 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.599212    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7cc85f3-afe1-4cff-aaa7-55c01da30af2" containerName="manager"
Jun 02 04:53:08.599280 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.599231    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7cc85f3-afe1-4cff-aaa7-55c01da30af2" containerName="manager"
Jun 02 04:53:08.599325 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.599312    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="f7cc85f3-afe1-4cff-aaa7-55c01da30af2" containerName="manager"
Jun 02 04:53:08.602549 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.602525    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.605388 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.605362    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-4kq9g\""
Jun 02 04:53:08.616033 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.616012    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"]
Jun 02 04:53:08.633124 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.633100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71c1c48e-df32-44e5-b88a-49bdb9380172-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.633243 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.633133    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c1c48e-df32-44e5-b88a-49bdb9380172-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.633243 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.633164    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmqcp\" (UniqueName: \"kubernetes.io/projected/71c1c48e-df32-44e5-b88a-49bdb9380172-kube-api-access-xmqcp\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.633243 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.633192    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71c1c48e-df32-44e5-b88a-49bdb9380172-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.633243 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.633223    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71c1c48e-df32-44e5-b88a-49bdb9380172-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.633420 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.633268    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71c1c48e-df32-44e5-b88a-49bdb9380172-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.633420 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.633293    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c1c48e-df32-44e5-b88a-49bdb9380172-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.633420 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.633307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71c1c48e-df32-44e5-b88a-49bdb9380172-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.633420 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.633328    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71c1c48e-df32-44e5-b88a-49bdb9380172-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.733992 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.733960    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71c1c48e-df32-44e5-b88a-49bdb9380172-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.734129 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.734001    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c1c48e-df32-44e5-b88a-49bdb9380172-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.734129 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.734021    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71c1c48e-df32-44e5-b88a-49bdb9380172-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.734129 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.734037    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71c1c48e-df32-44e5-b88a-49bdb9380172-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.734129 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.734059    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71c1c48e-df32-44e5-b88a-49bdb9380172-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.734284 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.734161    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c1c48e-df32-44e5-b88a-49bdb9380172-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.734284 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.734224    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xmqcp\" (UniqueName: \"kubernetes.io/projected/71c1c48e-df32-44e5-b88a-49bdb9380172-kube-api-access-xmqcp\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.734284 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.734275    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71c1c48e-df32-44e5-b88a-49bdb9380172-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.734438 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.734303    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71c1c48e-df32-44e5-b88a-49bdb9380172-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.734438 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.734422    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71c1c48e-df32-44e5-b88a-49bdb9380172-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.734532 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.734489    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c1c48e-df32-44e5-b88a-49bdb9380172-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.734587 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.734560    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c1c48e-df32-44e5-b88a-49bdb9380172-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.734685 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.734667    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71c1c48e-df32-44e5-b88a-49bdb9380172-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.734970 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.734948    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71c1c48e-df32-44e5-b88a-49bdb9380172-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.736643 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.736614    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71c1c48e-df32-44e5-b88a-49bdb9380172-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.736789 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.736772    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71c1c48e-df32-44e5-b88a-49bdb9380172-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.743473 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.743448    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71c1c48e-df32-44e5-b88a-49bdb9380172-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.744238 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.744215    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmqcp\" (UniqueName: \"kubernetes.io/projected/71c1c48e-df32-44e5-b88a-49bdb9380172-kube-api-access-xmqcp\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-wvxbc\" (UID: \"71c1c48e-df32-44e5-b88a-49bdb9380172\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:08.916559 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:08.916475    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:09.051362 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:09.051325    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"]
Jun 02 04:53:09.053888 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:53:09.053841    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71c1c48e_df32_44e5_b88a_49bdb9380172.slice/crio-e7a505d0708d293684a543902e7153dfd24e4cf5d93c30e0ad6a123c4e55d56a WatchSource:0}: Error finding container e7a505d0708d293684a543902e7153dfd24e4cf5d93c30e0ad6a123c4e55d56a: Status 404 returned error can't find the container with id e7a505d0708d293684a543902e7153dfd24e4cf5d93c30e0ad6a123c4e55d56a
Jun 02 04:53:09.056014 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:09.055985    2577 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jun 02 04:53:09.056113 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:09.056057    2577 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jun 02 04:53:09.056168 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:09.056120    2577 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jun 02 04:53:09.241883 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:09.241848    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc" event={"ID":"71c1c48e-df32-44e5-b88a-49bdb9380172","Type":"ContainerStarted","Data":"566a1c22f92ef598295f6f27805b3fb41e2343d3d7fe87cac304bb9b7787c0f0"}
Jun 02 04:53:09.242045 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:09.241893    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc" event={"ID":"71c1c48e-df32-44e5-b88a-49bdb9380172","Type":"ContainerStarted","Data":"e7a505d0708d293684a543902e7153dfd24e4cf5d93c30e0ad6a123c4e55d56a"}
Jun 02 04:53:09.263386 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:09.263330    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc" podStartSLOduration=1.263311531 podStartE2EDuration="1.263311531s" podCreationTimestamp="2026-06-02 04:53:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 04:53:09.262342451 +0000 UTC m=+636.303273451" watchObservedRunningTime="2026-06-02 04:53:09.263311531 +0000 UTC m=+636.304242504"
Jun 02 04:53:09.917593 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:09.917555    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:10.921968 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:10.921939    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:11.249076 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:11.249045    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:11.249796 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:11.249779    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-wvxbc"
Jun 02 04:53:12.556107 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:12.556074    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-p6sdb"]
Jun 02 04:53:12.559477 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:12.559456    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb"
Jun 02 04:53:12.561995 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:12.561972    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jun 02 04:53:12.562081 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:12.562014    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vjddx\""
Jun 02 04:53:12.568959 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:12.568936    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-p6sdb"]
Jun 02 04:53:12.660074 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:12.660044    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-p6sdb"]
Jun 02 04:53:12.669587 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:12.669545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4-config-file\") pod \"limitador-limitador-7d549b5b-p6sdb\" (UID: \"8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4\") " pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb"
Jun 02 04:53:12.669734 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:12.669696    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdpl8\" (UniqueName: \"kubernetes.io/projected/8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4-kube-api-access-zdpl8\") pod \"limitador-limitador-7d549b5b-p6sdb\" (UID: \"8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4\") " pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb"
Jun 02 04:53:12.770613 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:12.770584    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zdpl8\" (UniqueName: \"kubernetes.io/projected/8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4-kube-api-access-zdpl8\") pod \"limitador-limitador-7d549b5b-p6sdb\" (UID: \"8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4\") " pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb"
Jun 02 04:53:12.770744 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:12.770617    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4-config-file\") pod \"limitador-limitador-7d549b5b-p6sdb\" (UID: \"8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4\") " pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb"
Jun 02 04:53:12.771148 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:12.771131    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4-config-file\") pod \"limitador-limitador-7d549b5b-p6sdb\" (UID: \"8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4\") " pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb"
Jun 02 04:53:12.779379 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:12.779346    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdpl8\" (UniqueName: \"kubernetes.io/projected/8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4-kube-api-access-zdpl8\") pod \"limitador-limitador-7d549b5b-p6sdb\" (UID: \"8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4\") " pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb"
Jun 02 04:53:12.870299 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:12.870236    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb"
Jun 02 04:53:12.998153 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:12.998117    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-p6sdb"]
Jun 02 04:53:13.001032 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:53:13.001005    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eabdde2_86ff_49fc_9cf7_a3b0703fb0c4.slice/crio-d897be4cf66b797a01bf54a3df717b660faaf6a83f34abf21a23ee422d5112ea WatchSource:0}: Error finding container d897be4cf66b797a01bf54a3df717b660faaf6a83f34abf21a23ee422d5112ea: Status 404 returned error can't find the container with id d897be4cf66b797a01bf54a3df717b660faaf6a83f34abf21a23ee422d5112ea
Jun 02 04:53:13.256509 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.256475    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb" event={"ID":"8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4","Type":"ContainerStarted","Data":"d897be4cf66b797a01bf54a3df717b660faaf6a83f34abf21a23ee422d5112ea"}
Jun 02 04:53:13.377352 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.377327    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-4gwfx"]
Jun 02 04:53:13.382153 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.382134    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-4gwfx"
Jun 02 04:53:13.384487 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.384466    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-t6znp\""
Jun 02 04:53:13.384804 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.384786    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-4gwfx"]
Jun 02 04:53:13.477086 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.477062    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xb7w\" (UniqueName: \"kubernetes.io/projected/2197f3af-ca39-4ff6-b390-302bedf8b81e-kube-api-access-2xb7w\") pod \"authorino-f99f4b5cd-4gwfx\" (UID: \"2197f3af-ca39-4ff6-b390-302bedf8b81e\") " pod="kuadrant-system/authorino-f99f4b5cd-4gwfx"
Jun 02 04:53:13.564310 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.564258    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-lp5k6"]
Jun 02 04:53:13.567503 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.567487    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-lp5k6"
Jun 02 04:53:13.575012 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.574988    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-lp5k6"]
Jun 02 04:53:13.577785 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.577764    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2xb7w\" (UniqueName: \"kubernetes.io/projected/2197f3af-ca39-4ff6-b390-302bedf8b81e-kube-api-access-2xb7w\") pod \"authorino-f99f4b5cd-4gwfx\" (UID: \"2197f3af-ca39-4ff6-b390-302bedf8b81e\") " pod="kuadrant-system/authorino-f99f4b5cd-4gwfx"
Jun 02 04:53:13.587519 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.587498    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xb7w\" (UniqueName: \"kubernetes.io/projected/2197f3af-ca39-4ff6-b390-302bedf8b81e-kube-api-access-2xb7w\") pod \"authorino-f99f4b5cd-4gwfx\" (UID: \"2197f3af-ca39-4ff6-b390-302bedf8b81e\") " pod="kuadrant-system/authorino-f99f4b5cd-4gwfx"
Jun 02 04:53:13.678521 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.678447    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqsnd\" (UniqueName: \"kubernetes.io/projected/2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1-kube-api-access-lqsnd\") pod \"authorino-7498df8756-lp5k6\" (UID: \"2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1\") " pod="kuadrant-system/authorino-7498df8756-lp5k6"
Jun 02 04:53:13.692418 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.692392    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-4gwfx"
Jun 02 04:53:13.780208 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.779992    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lqsnd\" (UniqueName: \"kubernetes.io/projected/2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1-kube-api-access-lqsnd\") pod \"authorino-7498df8756-lp5k6\" (UID: \"2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1\") " pod="kuadrant-system/authorino-7498df8756-lp5k6"
Jun 02 04:53:13.792783 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.792714    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqsnd\" (UniqueName: \"kubernetes.io/projected/2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1-kube-api-access-lqsnd\") pod \"authorino-7498df8756-lp5k6\" (UID: \"2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1\") " pod="kuadrant-system/authorino-7498df8756-lp5k6"
Jun 02 04:53:13.858261 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.858136    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-4gwfx"]
Jun 02 04:53:13.877384 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:13.877033    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-lp5k6"
Jun 02 04:53:14.064146 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:14.064105    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-lp5k6"]
Jun 02 04:53:14.066763 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:53:14.066730    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f6d7bd0_d6f5_4a34_ba48_2bde507a98d1.slice/crio-cb39fcbc9394abbe3a8ed3120b31100b3ad411b278c680a5a5e28d2ed6c42266 WatchSource:0}: Error finding container cb39fcbc9394abbe3a8ed3120b31100b3ad411b278c680a5a5e28d2ed6c42266: Status 404 returned error can't find the container with id cb39fcbc9394abbe3a8ed3120b31100b3ad411b278c680a5a5e28d2ed6c42266
Jun 02 04:53:14.261366 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:14.261331    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-lp5k6" event={"ID":"2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1","Type":"ContainerStarted","Data":"cb39fcbc9394abbe3a8ed3120b31100b3ad411b278c680a5a5e28d2ed6c42266"}
Jun 02 04:53:14.262783 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:14.262756    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-4gwfx" event={"ID":"2197f3af-ca39-4ff6-b390-302bedf8b81e","Type":"ContainerStarted","Data":"2c06b40076f5c885ef30fff6b6912721a6b119739926994090b79749000aa47c"}
Jun 02 04:53:18.277826 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:18.277777    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-lp5k6" event={"ID":"2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1","Type":"ContainerStarted","Data":"2652833ae77205e72cf8c26ea6b64abd1b630d3d0761671f78e8d08cb1893fc6"}
Jun 02 04:53:18.279124 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:18.279099    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb" event={"ID":"8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4","Type":"ContainerStarted","Data":"a6146e152197d3659c5574f9268b0fdb7f8ced880552ef22c789327fcbee33da"}
Jun 02 04:53:18.279225 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:18.279189    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb"
Jun 02 04:53:18.280245 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:18.280226    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-4gwfx" event={"ID":"2197f3af-ca39-4ff6-b390-302bedf8b81e","Type":"ContainerStarted","Data":"f51378d0e461996737ad0e8e118bb30f61f95b64757c06bcfe15858a696c6a49"}
Jun 02 04:53:18.294991 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:18.294951    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-lp5k6" podStartSLOduration=1.706762407 podStartE2EDuration="5.294939898s" podCreationTimestamp="2026-06-02 04:53:13 +0000 UTC" firstStartedPulling="2026-06-02 04:53:14.068445743 +0000 UTC m=+641.109376707" lastFinishedPulling="2026-06-02 04:53:17.656623232 +0000 UTC m=+644.697554198" observedRunningTime="2026-06-02 04:53:18.293148606 +0000 UTC m=+645.334079578" watchObservedRunningTime="2026-06-02 04:53:18.294939898 +0000 UTC m=+645.335870890"
Jun 02 04:53:18.309160 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:18.309104    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb" podStartSLOduration=1.660837404 podStartE2EDuration="6.309088716s" podCreationTimestamp="2026-06-02 04:53:12 +0000 UTC" firstStartedPulling="2026-06-02 04:53:13.003022855 +0000 UTC m=+640.043953809" lastFinishedPulling="2026-06-02 04:53:17.651274166 +0000 UTC m=+644.692205121" observedRunningTime="2026-06-02 04:53:18.307585307 +0000 UTC m=+645.348516279" watchObservedRunningTime="2026-06-02 04:53:18.309088716 +0000 UTC m=+645.350019691"
Jun 02 04:53:18.321264 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:18.321240    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-4gwfx"]
Jun 02 04:53:18.330699 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:18.330664    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-4gwfx" podStartSLOduration=1.5921429809999998 podStartE2EDuration="5.330653778s" podCreationTimestamp="2026-06-02 04:53:13 +0000 UTC" firstStartedPulling="2026-06-02 04:53:13.865948816 +0000 UTC m=+640.906879770" lastFinishedPulling="2026-06-02 04:53:17.604459612 +0000 UTC m=+644.645390567" observedRunningTime="2026-06-02 04:53:18.329511215 +0000 UTC m=+645.370442186" watchObservedRunningTime="2026-06-02 04:53:18.330653778 +0000 UTC m=+645.371584749"
Jun 02 04:53:20.286779 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:20.286728    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-4gwfx" podUID="2197f3af-ca39-4ff6-b390-302bedf8b81e" containerName="authorino" containerID="cri-o://f51378d0e461996737ad0e8e118bb30f61f95b64757c06bcfe15858a696c6a49" gracePeriod=30
Jun 02 04:53:20.526831 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:20.526794    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-4gwfx"
Jun 02 04:53:20.644269 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:20.644206    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xb7w\" (UniqueName: \"kubernetes.io/projected/2197f3af-ca39-4ff6-b390-302bedf8b81e-kube-api-access-2xb7w\") pod \"2197f3af-ca39-4ff6-b390-302bedf8b81e\" (UID: \"2197f3af-ca39-4ff6-b390-302bedf8b81e\") "
Jun 02 04:53:20.646000 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:20.645980    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2197f3af-ca39-4ff6-b390-302bedf8b81e-kube-api-access-2xb7w" (OuterVolumeSpecName: "kube-api-access-2xb7w") pod "2197f3af-ca39-4ff6-b390-302bedf8b81e" (UID: "2197f3af-ca39-4ff6-b390-302bedf8b81e"). InnerVolumeSpecName "kube-api-access-2xb7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 04:53:20.745678 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:20.745644    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xb7w\" (UniqueName: \"kubernetes.io/projected/2197f3af-ca39-4ff6-b390-302bedf8b81e-kube-api-access-2xb7w\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:53:21.291027 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:21.290992    2577 generic.go:358] "Generic (PLEG): container finished" podID="2197f3af-ca39-4ff6-b390-302bedf8b81e" containerID="f51378d0e461996737ad0e8e118bb30f61f95b64757c06bcfe15858a696c6a49" exitCode=0
Jun 02 04:53:21.291412 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:21.291040    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-4gwfx"
Jun 02 04:53:21.291412 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:21.291077    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-4gwfx" event={"ID":"2197f3af-ca39-4ff6-b390-302bedf8b81e","Type":"ContainerDied","Data":"f51378d0e461996737ad0e8e118bb30f61f95b64757c06bcfe15858a696c6a49"}
Jun 02 04:53:21.291412 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:21.291115    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-4gwfx" event={"ID":"2197f3af-ca39-4ff6-b390-302bedf8b81e","Type":"ContainerDied","Data":"2c06b40076f5c885ef30fff6b6912721a6b119739926994090b79749000aa47c"}
Jun 02 04:53:21.291412 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:21.291133    2577 scope.go:117] "RemoveContainer" containerID="f51378d0e461996737ad0e8e118bb30f61f95b64757c06bcfe15858a696c6a49"
Jun 02 04:53:21.303231 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:21.303212    2577 scope.go:117] "RemoveContainer" containerID="f51378d0e461996737ad0e8e118bb30f61f95b64757c06bcfe15858a696c6a49"
Jun 02 04:53:21.303473 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:53:21.303458    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f51378d0e461996737ad0e8e118bb30f61f95b64757c06bcfe15858a696c6a49\": container with ID starting with f51378d0e461996737ad0e8e118bb30f61f95b64757c06bcfe15858a696c6a49 not found: ID does not exist" containerID="f51378d0e461996737ad0e8e118bb30f61f95b64757c06bcfe15858a696c6a49"
Jun 02 04:53:21.303515 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:21.303480    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f51378d0e461996737ad0e8e118bb30f61f95b64757c06bcfe15858a696c6a49"} err="failed to get container status \"f51378d0e461996737ad0e8e118bb30f61f95b64757c06bcfe15858a696c6a49\": rpc error: code = NotFound desc = could not find container \"f51378d0e461996737ad0e8e118bb30f61f95b64757c06bcfe15858a696c6a49\": container with ID starting with f51378d0e461996737ad0e8e118bb30f61f95b64757c06bcfe15858a696c6a49 not found: ID does not exist"
Jun 02 04:53:21.315370 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:21.315348    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-4gwfx"]
Jun 02 04:53:21.319182 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:21.319162    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-4gwfx"]
Jun 02 04:53:21.444344 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:21.444313    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2197f3af-ca39-4ff6-b390-302bedf8b81e" path="/var/lib/kubelet/pods/2197f3af-ca39-4ff6-b390-302bedf8b81e/volumes"
Jun 02 04:53:27.951877 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:27.951838    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-p6sdb"]
Jun 02 04:53:27.952345 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:27.952134    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb" podUID="8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4" containerName="limitador" containerID="cri-o://a6146e152197d3659c5574f9268b0fdb7f8ced880552ef22c789327fcbee33da" gracePeriod=30
Jun 02 04:53:27.952790 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:27.952773    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb"
Jun 02 04:53:28.317346 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:28.317323    2577 generic.go:358] "Generic (PLEG): container finished" podID="8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4" containerID="a6146e152197d3659c5574f9268b0fdb7f8ced880552ef22c789327fcbee33da" exitCode=0
Jun 02 04:53:28.317453 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:28.317404    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb" event={"ID":"8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4","Type":"ContainerDied","Data":"a6146e152197d3659c5574f9268b0fdb7f8ced880552ef22c789327fcbee33da"}
Jun 02 04:53:28.901181 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:28.901160    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb"
Jun 02 04:53:29.006236 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:29.006213    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4-config-file\") pod \"8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4\" (UID: \"8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4\") "
Jun 02 04:53:29.006642 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:29.006286    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdpl8\" (UniqueName: \"kubernetes.io/projected/8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4-kube-api-access-zdpl8\") pod \"8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4\" (UID: \"8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4\") "
Jun 02 04:53:29.006642 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:29.006622    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4-config-file" (OuterVolumeSpecName: "config-file") pod "8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4" (UID: "8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 04:53:29.008422 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:29.008396    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4-kube-api-access-zdpl8" (OuterVolumeSpecName: "kube-api-access-zdpl8") pod "8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4" (UID: "8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4"). InnerVolumeSpecName "kube-api-access-zdpl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 04:53:29.107157 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:29.107092    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdpl8\" (UniqueName: \"kubernetes.io/projected/8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4-kube-api-access-zdpl8\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:53:29.107157 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:29.107116    2577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4-config-file\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:53:29.322050 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:29.322020    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb" event={"ID":"8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4","Type":"ContainerDied","Data":"d897be4cf66b797a01bf54a3df717b660faaf6a83f34abf21a23ee422d5112ea"}
Jun 02 04:53:29.322173 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:29.322069    2577 scope.go:117] "RemoveContainer" containerID="a6146e152197d3659c5574f9268b0fdb7f8ced880552ef22c789327fcbee33da"
Jun 02 04:53:29.322173 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:29.322076    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-p6sdb"
Jun 02 04:53:29.346064 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:29.346039    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-p6sdb"]
Jun 02 04:53:29.350945 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:29.350925    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-p6sdb"]
Jun 02 04:53:29.444892 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:29.444841    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4" path="/var/lib/kubelet/pods/8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4/volumes"
Jun 02 04:53:33.858478 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:33.858447    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-sng54"]
Jun 02 04:53:33.858978 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:33.858959    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4" containerName="limitador"
Jun 02 04:53:33.859050 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:33.858980    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4" containerName="limitador"
Jun 02 04:53:33.859050 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:33.859002    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2197f3af-ca39-4ff6-b390-302bedf8b81e" containerName="authorino"
Jun 02 04:53:33.859050 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:33.859012    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2197f3af-ca39-4ff6-b390-302bedf8b81e" containerName="authorino"
Jun 02 04:53:33.859196 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:33.859119    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="8eabdde2-86ff-49fc-9cf7-a3b0703fb0c4" containerName="limitador"
Jun 02 04:53:33.859196 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:33.859136    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="2197f3af-ca39-4ff6-b390-302bedf8b81e" containerName="authorino"
Jun 02 04:53:33.863573 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:33.863555    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-sng54"
Jun 02 04:53:33.866204 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:33.866182    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
Jun 02 04:53:33.871479 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:33.871440    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-sng54"]
Jun 02 04:53:33.945099 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:33.945075    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ba819424-07f5-4215-98d7-534d6c678735-data\") pod \"postgres-68c4fbbd6f-sng54\" (UID: \"ba819424-07f5-4215-98d7-534d6c678735\") " pod="opendatahub/postgres-68c4fbbd6f-sng54"
Jun 02 04:53:33.945214 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:33.945115    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnbcr\" (UniqueName: \"kubernetes.io/projected/ba819424-07f5-4215-98d7-534d6c678735-kube-api-access-hnbcr\") pod \"postgres-68c4fbbd6f-sng54\" (UID: \"ba819424-07f5-4215-98d7-534d6c678735\") " pod="opendatahub/postgres-68c4fbbd6f-sng54"
Jun 02 04:53:34.046505 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:34.046476    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ba819424-07f5-4215-98d7-534d6c678735-data\") pod \"postgres-68c4fbbd6f-sng54\" (UID: \"ba819424-07f5-4215-98d7-534d6c678735\") " pod="opendatahub/postgres-68c4fbbd6f-sng54"
Jun 02 04:53:34.046505 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:34.046508    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hnbcr\" (UniqueName: \"kubernetes.io/projected/ba819424-07f5-4215-98d7-534d6c678735-kube-api-access-hnbcr\") pod \"postgres-68c4fbbd6f-sng54\" (UID: \"ba819424-07f5-4215-98d7-534d6c678735\") " pod="opendatahub/postgres-68c4fbbd6f-sng54"
Jun 02 04:53:34.046851 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:34.046807    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ba819424-07f5-4215-98d7-534d6c678735-data\") pod \"postgres-68c4fbbd6f-sng54\" (UID: \"ba819424-07f5-4215-98d7-534d6c678735\") " pod="opendatahub/postgres-68c4fbbd6f-sng54"
Jun 02 04:53:34.055530 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:34.055500    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnbcr\" (UniqueName: \"kubernetes.io/projected/ba819424-07f5-4215-98d7-534d6c678735-kube-api-access-hnbcr\") pod \"postgres-68c4fbbd6f-sng54\" (UID: \"ba819424-07f5-4215-98d7-534d6c678735\") " pod="opendatahub/postgres-68c4fbbd6f-sng54"
Jun 02 04:53:34.175274 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:34.175218    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-sng54"
Jun 02 04:53:34.304947 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:34.304923    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-sng54"]
Jun 02 04:53:34.307407 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:53:34.307378    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba819424_07f5_4215_98d7_534d6c678735.slice/crio-3a01fdf697ca8678c8a805784ad245bee24ea749f14bf4b948cf5284208ef3a9 WatchSource:0}: Error finding container 3a01fdf697ca8678c8a805784ad245bee24ea749f14bf4b948cf5284208ef3a9: Status 404 returned error can't find the container with id 3a01fdf697ca8678c8a805784ad245bee24ea749f14bf4b948cf5284208ef3a9
Jun 02 04:53:34.340293 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:34.340269    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-sng54" event={"ID":"ba819424-07f5-4215-98d7-534d6c678735","Type":"ContainerStarted","Data":"3a01fdf697ca8678c8a805784ad245bee24ea749f14bf4b948cf5284208ef3a9"}
Jun 02 04:53:37.352960 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:37.352929    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-sng54" event={"ID":"ba819424-07f5-4215-98d7-534d6c678735","Type":"ContainerStarted","Data":"43b8e6322a636c73b031ec04706524254c3771defead075acba1b269fa66952f"}
Jun 02 04:53:37.353300 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:37.353155    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-sng54"
Jun 02 04:53:37.370865 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:37.370804    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-sng54" podStartSLOduration=2.036540062 podStartE2EDuration="4.370792675s" podCreationTimestamp="2026-06-02 04:53:33 +0000 UTC" firstStartedPulling="2026-06-02 04:53:34.308695896 +0000 UTC m=+661.349626846" lastFinishedPulling="2026-06-02 04:53:36.642948509 +0000 UTC m=+663.683879459" observedRunningTime="2026-06-02 04:53:37.369724535 +0000 UTC m=+664.410655508" watchObservedRunningTime="2026-06-02 04:53:37.370792675 +0000 UTC m=+664.411723646"
Jun 02 04:53:43.388235 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:43.388209    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-sng54"
Jun 02 04:53:53.246827 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:53.246786    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/keycloak-operator-7864d68cbd-p55hl"]
Jun 02 04:53:53.259081 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:53.259054    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-7864d68cbd-p55hl"]
Jun 02 04:53:53.259213 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:53.259151    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-7864d68cbd-p55hl"
Jun 02 04:53:53.262137 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:53.262113    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\""
Jun 02 04:53:53.263354 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:53.263337    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"keycloak-operator-dockercfg-7k7md\""
Jun 02 04:53:53.263485 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:53.263337    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\""
Jun 02 04:53:53.294410 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:53.294385    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l7qg\" (UniqueName: \"kubernetes.io/projected/1e292e46-fbd0-40dc-a341-3b560d389754-kube-api-access-5l7qg\") pod \"keycloak-operator-7864d68cbd-p55hl\" (UID: \"1e292e46-fbd0-40dc-a341-3b560d389754\") " pod="keycloak-system/keycloak-operator-7864d68cbd-p55hl"
Jun 02 04:53:53.394993 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:53.394964    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5l7qg\" (UniqueName: \"kubernetes.io/projected/1e292e46-fbd0-40dc-a341-3b560d389754-kube-api-access-5l7qg\") pod \"keycloak-operator-7864d68cbd-p55hl\" (UID: \"1e292e46-fbd0-40dc-a341-3b560d389754\") " pod="keycloak-system/keycloak-operator-7864d68cbd-p55hl"
Jun 02 04:53:53.404008 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:53.403989    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l7qg\" (UniqueName: \"kubernetes.io/projected/1e292e46-fbd0-40dc-a341-3b560d389754-kube-api-access-5l7qg\") pod \"keycloak-operator-7864d68cbd-p55hl\" (UID: \"1e292e46-fbd0-40dc-a341-3b560d389754\") " pod="keycloak-system/keycloak-operator-7864d68cbd-p55hl"
Jun 02 04:53:53.570106 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:53.570038    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-7864d68cbd-p55hl"
Jun 02 04:53:53.694595 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:53.694571    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-7864d68cbd-p55hl"]
Jun 02 04:53:53.696725 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:53:53.696696    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e292e46_fbd0_40dc_a341_3b560d389754.slice/crio-d668f7e5bc4432089664ca0ac5d31c7b3aacbb196b6b2f0dcbc4e54d1df1c88a WatchSource:0}: Error finding container d668f7e5bc4432089664ca0ac5d31c7b3aacbb196b6b2f0dcbc4e54d1df1c88a: Status 404 returned error can't find the container with id d668f7e5bc4432089664ca0ac5d31c7b3aacbb196b6b2f0dcbc4e54d1df1c88a
Jun 02 04:53:54.413536 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:54.413492    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-7864d68cbd-p55hl" event={"ID":"1e292e46-fbd0-40dc-a341-3b560d389754","Type":"ContainerStarted","Data":"d668f7e5bc4432089664ca0ac5d31c7b3aacbb196b6b2f0dcbc4e54d1df1c88a"}
Jun 02 04:53:59.433754 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:59.433719    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-7864d68cbd-p55hl" event={"ID":"1e292e46-fbd0-40dc-a341-3b560d389754","Type":"ContainerStarted","Data":"2ce94baebef46d3e4bfcc4936e4abd5fd1df1768ae599ac1bbbf1b803587e4c2"}
Jun 02 04:53:59.452359 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:53:59.452317    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/keycloak-operator-7864d68cbd-p55hl" podStartSLOduration=0.823769019 podStartE2EDuration="6.452303367s" podCreationTimestamp="2026-06-02 04:53:53 +0000 UTC" firstStartedPulling="2026-06-02 04:53:53.698117149 +0000 UTC m=+680.739048098" lastFinishedPulling="2026-06-02 04:53:59.326651496 +0000 UTC m=+686.367582446" observedRunningTime="2026-06-02 04:53:59.451096719 +0000 UTC m=+686.492027691" watchObservedRunningTime="2026-06-02 04:53:59.452303367 +0000 UTC m=+686.493234338"
Jun 02 04:54:37.289205 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.289175    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-6wsz6"]
Jun 02 04:54:37.299414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.299388    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-6wsz6"
Jun 02 04:54:37.300856 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.300831    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-6wsz6"]
Jun 02 04:54:37.445572 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.445548    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pz6r\" (UniqueName: \"kubernetes.io/projected/521f8592-f410-4a51-98f1-594a008c4ae8-kube-api-access-9pz6r\") pod \"authorino-8b475cf9f-6wsz6\" (UID: \"521f8592-f410-4a51-98f1-594a008c4ae8\") " pod="kuadrant-system/authorino-8b475cf9f-6wsz6"
Jun 02 04:54:37.528284 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.528263    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-6wsz6"]
Jun 02 04:54:37.528445 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:54:37.528428    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-9pz6r], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-8b475cf9f-6wsz6" podUID="521f8592-f410-4a51-98f1-594a008c4ae8"
Jun 02 04:54:37.547025 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.546962    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9pz6r\" (UniqueName: \"kubernetes.io/projected/521f8592-f410-4a51-98f1-594a008c4ae8-kube-api-access-9pz6r\") pod \"authorino-8b475cf9f-6wsz6\" (UID: \"521f8592-f410-4a51-98f1-594a008c4ae8\") " pod="kuadrant-system/authorino-8b475cf9f-6wsz6"
Jun 02 04:54:37.557597 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.557575    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b6c858c6f-465pp"]
Jun 02 04:54:37.559499 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.559477    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pz6r\" (UniqueName: \"kubernetes.io/projected/521f8592-f410-4a51-98f1-594a008c4ae8-kube-api-access-9pz6r\") pod \"authorino-8b475cf9f-6wsz6\" (UID: \"521f8592-f410-4a51-98f1-594a008c4ae8\") " pod="kuadrant-system/authorino-8b475cf9f-6wsz6"
Jun 02 04:54:37.561010 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.560990    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6c858c6f-465pp"
Jun 02 04:54:37.562491 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.562473    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-6wsz6"
Jun 02 04:54:37.563407 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.563385    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jun 02 04:54:37.568008 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.567988    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b6c858c6f-465pp"]
Jun 02 04:54:37.584056 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.584041    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-6wsz6"
Jun 02 04:54:37.648313 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.648291    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/913564d1-5f60-43f9-b995-8a7f6f27f3b2-tls-cert\") pod \"authorino-b6c858c6f-465pp\" (UID: \"913564d1-5f60-43f9-b995-8a7f6f27f3b2\") " pod="kuadrant-system/authorino-b6c858c6f-465pp"
Jun 02 04:54:37.648402 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.648322    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82529\" (UniqueName: \"kubernetes.io/projected/913564d1-5f60-43f9-b995-8a7f6f27f3b2-kube-api-access-82529\") pod \"authorino-b6c858c6f-465pp\" (UID: \"913564d1-5f60-43f9-b995-8a7f6f27f3b2\") " pod="kuadrant-system/authorino-b6c858c6f-465pp"
Jun 02 04:54:37.749186 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.749165    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pz6r\" (UniqueName: \"kubernetes.io/projected/521f8592-f410-4a51-98f1-594a008c4ae8-kube-api-access-9pz6r\") pod \"521f8592-f410-4a51-98f1-594a008c4ae8\" (UID: \"521f8592-f410-4a51-98f1-594a008c4ae8\") "
Jun 02 04:54:37.749340 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.749325    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/913564d1-5f60-43f9-b995-8a7f6f27f3b2-tls-cert\") pod \"authorino-b6c858c6f-465pp\" (UID: \"913564d1-5f60-43f9-b995-8a7f6f27f3b2\") " pod="kuadrant-system/authorino-b6c858c6f-465pp"
Jun 02 04:54:37.749414 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.749348    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-82529\" (UniqueName: \"kubernetes.io/projected/913564d1-5f60-43f9-b995-8a7f6f27f3b2-kube-api-access-82529\") pod \"authorino-b6c858c6f-465pp\" (UID: \"913564d1-5f60-43f9-b995-8a7f6f27f3b2\") " pod="kuadrant-system/authorino-b6c858c6f-465pp"
Jun 02 04:54:37.751287 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.751264    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/521f8592-f410-4a51-98f1-594a008c4ae8-kube-api-access-9pz6r" (OuterVolumeSpecName: "kube-api-access-9pz6r") pod "521f8592-f410-4a51-98f1-594a008c4ae8" (UID: "521f8592-f410-4a51-98f1-594a008c4ae8"). InnerVolumeSpecName "kube-api-access-9pz6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 04:54:37.751609 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.751594    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/913564d1-5f60-43f9-b995-8a7f6f27f3b2-tls-cert\") pod \"authorino-b6c858c6f-465pp\" (UID: \"913564d1-5f60-43f9-b995-8a7f6f27f3b2\") " pod="kuadrant-system/authorino-b6c858c6f-465pp"
Jun 02 04:54:37.756765 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.756749    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82529\" (UniqueName: \"kubernetes.io/projected/913564d1-5f60-43f9-b995-8a7f6f27f3b2-kube-api-access-82529\") pod \"authorino-b6c858c6f-465pp\" (UID: \"913564d1-5f60-43f9-b995-8a7f6f27f3b2\") " pod="kuadrant-system/authorino-b6c858c6f-465pp"
Jun 02 04:54:37.850530 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.850477    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pz6r\" (UniqueName: \"kubernetes.io/projected/521f8592-f410-4a51-98f1-594a008c4ae8-kube-api-access-9pz6r\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:54:37.883554 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:37.883524    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6c858c6f-465pp"
Jun 02 04:54:38.010355 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:38.010329    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b6c858c6f-465pp"]
Jun 02 04:54:38.011622 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:54:38.011596    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod913564d1_5f60_43f9_b995_8a7f6f27f3b2.slice/crio-1fe9d0c7a1421c4995436a8fbfe69d79cadbe767254f261aecc2586d61e55745 WatchSource:0}: Error finding container 1fe9d0c7a1421c4995436a8fbfe69d79cadbe767254f261aecc2586d61e55745: Status 404 returned error can't find the container with id 1fe9d0c7a1421c4995436a8fbfe69d79cadbe767254f261aecc2586d61e55745
Jun 02 04:54:38.012905 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:38.012889    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 04:54:38.567965 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:38.567930    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6c858c6f-465pp" event={"ID":"913564d1-5f60-43f9-b995-8a7f6f27f3b2","Type":"ContainerStarted","Data":"d288e0a3d0f968992ef77ce1de1b9dac4eb42c91482a9d38f6f57da75fabcae8"}
Jun 02 04:54:38.567965 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:38.567973    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6c858c6f-465pp" event={"ID":"913564d1-5f60-43f9-b995-8a7f6f27f3b2","Type":"ContainerStarted","Data":"1fe9d0c7a1421c4995436a8fbfe69d79cadbe767254f261aecc2586d61e55745"}
Jun 02 04:54:38.568409 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:38.567946    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-6wsz6"
Jun 02 04:54:38.586200 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:38.586154    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-b6c858c6f-465pp" podStartSLOduration=1.248600597 podStartE2EDuration="1.58614173s" podCreationTimestamp="2026-06-02 04:54:37 +0000 UTC" firstStartedPulling="2026-06-02 04:54:38.013008387 +0000 UTC m=+725.053939337" lastFinishedPulling="2026-06-02 04:54:38.350549516 +0000 UTC m=+725.391480470" observedRunningTime="2026-06-02 04:54:38.584148073 +0000 UTC m=+725.625079046" watchObservedRunningTime="2026-06-02 04:54:38.58614173 +0000 UTC m=+725.627072702"
Jun 02 04:54:38.611041 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:38.610936    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-6wsz6"]
Jun 02 04:54:38.617655 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:38.617627    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-lp5k6"]
Jun 02 04:54:38.617846 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:38.617802    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-lp5k6" podUID="2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1" containerName="authorino" containerID="cri-o://2652833ae77205e72cf8c26ea6b64abd1b630d3d0761671f78e8d08cb1893fc6" gracePeriod=30
Jun 02 04:54:38.620241 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:38.620218    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-6wsz6"]
Jun 02 04:54:38.860843 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:38.860799    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-lp5k6"
Jun 02 04:54:38.959629 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:38.959570    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqsnd\" (UniqueName: \"kubernetes.io/projected/2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1-kube-api-access-lqsnd\") pod \"2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1\" (UID: \"2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1\") "
Jun 02 04:54:38.961504 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:38.961485    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1-kube-api-access-lqsnd" (OuterVolumeSpecName: "kube-api-access-lqsnd") pod "2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1" (UID: "2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1"). InnerVolumeSpecName "kube-api-access-lqsnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 04:54:39.060970 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:39.060946    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqsnd\" (UniqueName: \"kubernetes.io/projected/2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1-kube-api-access-lqsnd\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:54:39.446568 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:39.446538    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="521f8592-f410-4a51-98f1-594a008c4ae8" path="/var/lib/kubelet/pods/521f8592-f410-4a51-98f1-594a008c4ae8/volumes"
Jun 02 04:54:39.571738 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:39.571714    2577 generic.go:358] "Generic (PLEG): container finished" podID="2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1" containerID="2652833ae77205e72cf8c26ea6b64abd1b630d3d0761671f78e8d08cb1893fc6" exitCode=0
Jun 02 04:54:39.572087 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:39.571800    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-lp5k6"
Jun 02 04:54:39.572087 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:39.571841    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-lp5k6" event={"ID":"2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1","Type":"ContainerDied","Data":"2652833ae77205e72cf8c26ea6b64abd1b630d3d0761671f78e8d08cb1893fc6"}
Jun 02 04:54:39.572087 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:39.571881    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-lp5k6" event={"ID":"2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1","Type":"ContainerDied","Data":"cb39fcbc9394abbe3a8ed3120b31100b3ad411b278c680a5a5e28d2ed6c42266"}
Jun 02 04:54:39.572087 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:39.571898    2577 scope.go:117] "RemoveContainer" containerID="2652833ae77205e72cf8c26ea6b64abd1b630d3d0761671f78e8d08cb1893fc6"
Jun 02 04:54:39.579635 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:39.579618    2577 scope.go:117] "RemoveContainer" containerID="2652833ae77205e72cf8c26ea6b64abd1b630d3d0761671f78e8d08cb1893fc6"
Jun 02 04:54:39.579869 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:54:39.579851    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2652833ae77205e72cf8c26ea6b64abd1b630d3d0761671f78e8d08cb1893fc6\": container with ID starting with 2652833ae77205e72cf8c26ea6b64abd1b630d3d0761671f78e8d08cb1893fc6 not found: ID does not exist" containerID="2652833ae77205e72cf8c26ea6b64abd1b630d3d0761671f78e8d08cb1893fc6"
Jun 02 04:54:39.579923 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:39.579879    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2652833ae77205e72cf8c26ea6b64abd1b630d3d0761671f78e8d08cb1893fc6"} err="failed to get container status \"2652833ae77205e72cf8c26ea6b64abd1b630d3d0761671f78e8d08cb1893fc6\": rpc error: code = NotFound desc = could not find container \"2652833ae77205e72cf8c26ea6b64abd1b630d3d0761671f78e8d08cb1893fc6\": container with ID starting with 2652833ae77205e72cf8c26ea6b64abd1b630d3d0761671f78e8d08cb1893fc6 not found: ID does not exist"
Jun 02 04:54:39.589876 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:39.589851    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-lp5k6"]
Jun 02 04:54:39.591517 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:39.591501    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-lp5k6"]
Jun 02 04:54:41.444074 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:41.444042    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1" path="/var/lib/kubelet/pods/2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1/volumes"
Jun 02 04:54:42.427392 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:42.427357    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-679b7f88b7-dzdbh"]
Jun 02 04:54:42.427702 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:42.427690    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1" containerName="authorino"
Jun 02 04:54:42.427743 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:42.427703    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1" containerName="authorino"
Jun 02 04:54:42.427777 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:42.427753    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="2f6d7bd0-d6f5-4a34-ba48-2bde507a98d1" containerName="authorino"
Jun 02 04:54:42.431914 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:42.431894    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-679b7f88b7-dzdbh"
Jun 02 04:54:42.434782 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:42.434761    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jun 02 04:54:42.434852 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:42.434829    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jdfgh\""
Jun 02 04:54:42.440329 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:42.440308    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-679b7f88b7-dzdbh"]
Jun 02 04:54:42.586892 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:42.586867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgsgw\" (UniqueName: \"kubernetes.io/projected/2b03f4b1-2cac-4a56-9d30-5cb87927c426-kube-api-access-zgsgw\") pod \"maas-controller-679b7f88b7-dzdbh\" (UID: \"2b03f4b1-2cac-4a56-9d30-5cb87927c426\") " pod="opendatahub/maas-controller-679b7f88b7-dzdbh"
Jun 02 04:54:42.687286 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:42.687226    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zgsgw\" (UniqueName: \"kubernetes.io/projected/2b03f4b1-2cac-4a56-9d30-5cb87927c426-kube-api-access-zgsgw\") pod \"maas-controller-679b7f88b7-dzdbh\" (UID: \"2b03f4b1-2cac-4a56-9d30-5cb87927c426\") " pod="opendatahub/maas-controller-679b7f88b7-dzdbh"
Jun 02 04:54:42.696796 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:42.696779    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgsgw\" (UniqueName: \"kubernetes.io/projected/2b03f4b1-2cac-4a56-9d30-5cb87927c426-kube-api-access-zgsgw\") pod \"maas-controller-679b7f88b7-dzdbh\" (UID: \"2b03f4b1-2cac-4a56-9d30-5cb87927c426\") " pod="opendatahub/maas-controller-679b7f88b7-dzdbh"
Jun 02 04:54:42.742642 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:42.742617    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-679b7f88b7-dzdbh"
Jun 02 04:54:42.867559 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:42.867538    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-679b7f88b7-dzdbh"]
Jun 02 04:54:42.869693 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:54:42.869664    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b03f4b1_2cac_4a56_9d30_5cb87927c426.slice/crio-3a5d14dd301d1170d5bbfd6a8afcfaf22defbb9af14080a51297026f78e52562 WatchSource:0}: Error finding container 3a5d14dd301d1170d5bbfd6a8afcfaf22defbb9af14080a51297026f78e52562: Status 404 returned error can't find the container with id 3a5d14dd301d1170d5bbfd6a8afcfaf22defbb9af14080a51297026f78e52562
Jun 02 04:54:43.588132 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:43.588096    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-679b7f88b7-dzdbh" event={"ID":"2b03f4b1-2cac-4a56-9d30-5cb87927c426","Type":"ContainerStarted","Data":"3a5d14dd301d1170d5bbfd6a8afcfaf22defbb9af14080a51297026f78e52562"}
Jun 02 04:54:45.595606 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:45.595577    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-679b7f88b7-dzdbh" event={"ID":"2b03f4b1-2cac-4a56-9d30-5cb87927c426","Type":"ContainerStarted","Data":"34ca32e9f318fd1297191c5a6e6365c49d19a26e6dd1ec9fce7eec4de7905e85"}
Jun 02 04:54:45.595978 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:45.595688    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-679b7f88b7-dzdbh"
Jun 02 04:54:45.614410 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:45.614366    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-679b7f88b7-dzdbh" podStartSLOduration=1.358418074 podStartE2EDuration="3.614354945s" podCreationTimestamp="2026-06-02 04:54:42 +0000 UTC" firstStartedPulling="2026-06-02 04:54:42.871012342 +0000 UTC m=+729.911943292" lastFinishedPulling="2026-06-02 04:54:45.126949211 +0000 UTC m=+732.167880163" observedRunningTime="2026-06-02 04:54:45.61352078 +0000 UTC m=+732.654451751" watchObservedRunningTime="2026-06-02 04:54:45.614354945 +0000 UTC m=+732.655285916"
Jun 02 04:54:56.605895 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:54:56.605860    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-679b7f88b7-dzdbh"
Jun 02 04:56:07.140603 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:07.140525    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-669dbdf959-9btr2"]
Jun 02 04:56:07.143975 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:07.143955    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-669dbdf959-9btr2"
Jun 02 04:56:07.146461 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:07.146443    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\""
Jun 02 04:56:07.151759 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:07.151734    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-669dbdf959-9btr2"]
Jun 02 04:56:07.238955 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:07.238934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-tls-cert\") pod \"authorino-669dbdf959-9btr2\" (UID: \"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae\") " pod="kuadrant-system/authorino-669dbdf959-9btr2"
Jun 02 04:56:07.239048 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:07.238970    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-oidc-ca\") pod \"authorino-669dbdf959-9btr2\" (UID: \"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae\") " pod="kuadrant-system/authorino-669dbdf959-9btr2"
Jun 02 04:56:07.239048 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:07.239037    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhvqv\" (UniqueName: \"kubernetes.io/projected/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-kube-api-access-zhvqv\") pod \"authorino-669dbdf959-9btr2\" (UID: \"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae\") " pod="kuadrant-system/authorino-669dbdf959-9btr2"
Jun 02 04:56:07.339377 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:07.339353    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-tls-cert\") pod \"authorino-669dbdf959-9btr2\" (UID: \"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae\") " pod="kuadrant-system/authorino-669dbdf959-9btr2"
Jun 02 04:56:07.339469 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:07.339386    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-oidc-ca\") pod \"authorino-669dbdf959-9btr2\" (UID: \"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae\") " pod="kuadrant-system/authorino-669dbdf959-9btr2"
Jun 02 04:56:07.339469 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:07.339422    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zhvqv\" (UniqueName: \"kubernetes.io/projected/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-kube-api-access-zhvqv\") pod \"authorino-669dbdf959-9btr2\" (UID: \"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae\") " pod="kuadrant-system/authorino-669dbdf959-9btr2"
Jun 02 04:56:07.339990 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:07.339972    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-oidc-ca\") pod \"authorino-669dbdf959-9btr2\" (UID: \"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae\") " pod="kuadrant-system/authorino-669dbdf959-9btr2"
Jun 02 04:56:07.341664 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:07.341644    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-tls-cert\") pod \"authorino-669dbdf959-9btr2\" (UID: \"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae\") " pod="kuadrant-system/authorino-669dbdf959-9btr2"
Jun 02 04:56:07.348755 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:07.348733    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhvqv\" (UniqueName: \"kubernetes.io/projected/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-kube-api-access-zhvqv\") pod \"authorino-669dbdf959-9btr2\" (UID: \"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae\") " pod="kuadrant-system/authorino-669dbdf959-9btr2"
Jun 02 04:56:07.453451 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:07.453402    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-669dbdf959-9btr2"
Jun 02 04:56:07.579618 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:07.579586    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-669dbdf959-9btr2"]
Jun 02 04:56:07.581752 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:56:07.581725    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d30e8bd_f3ef_4480_a62d_c9602fdd92ae.slice/crio-fdd3964a1482441cdfcba7a2903a72d475376d19806d9665c213ae76b94638b5 WatchSource:0}: Error finding container fdd3964a1482441cdfcba7a2903a72d475376d19806d9665c213ae76b94638b5: Status 404 returned error can't find the container with id fdd3964a1482441cdfcba7a2903a72d475376d19806d9665c213ae76b94638b5
Jun 02 04:56:07.879208 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:07.879167    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-669dbdf959-9btr2" event={"ID":"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae","Type":"ContainerStarted","Data":"fdd3964a1482441cdfcba7a2903a72d475376d19806d9665c213ae76b94638b5"}
Jun 02 04:56:08.884011 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:08.883980    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-669dbdf959-9btr2" event={"ID":"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae","Type":"ContainerStarted","Data":"d8998d03788047acdcbee888c241ba60f664093ac449387d5fe55d9f9d962154"}
Jun 02 04:56:08.904412 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:08.904367    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-669dbdf959-9btr2" podStartSLOduration=1.559077569 podStartE2EDuration="1.904353953s" podCreationTimestamp="2026-06-02 04:56:07 +0000 UTC" firstStartedPulling="2026-06-02 04:56:07.582946873 +0000 UTC m=+814.623877826" lastFinishedPulling="2026-06-02 04:56:07.928223226 +0000 UTC m=+814.969154210" observedRunningTime="2026-06-02 04:56:08.903919852 +0000 UTC m=+815.944850825" watchObservedRunningTime="2026-06-02 04:56:08.904353953 +0000 UTC m=+815.945284924"
Jun 02 04:56:08.931892 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:08.931863    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b6c858c6f-465pp"]
Jun 02 04:56:08.932129 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:08.932047    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-b6c858c6f-465pp" podUID="913564d1-5f60-43f9-b995-8a7f6f27f3b2" containerName="authorino" containerID="cri-o://d288e0a3d0f968992ef77ce1de1b9dac4eb42c91482a9d38f6f57da75fabcae8" gracePeriod=30
Jun 02 04:56:09.180865 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:09.180842    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6c858c6f-465pp"
Jun 02 04:56:09.251768 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:09.251740    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82529\" (UniqueName: \"kubernetes.io/projected/913564d1-5f60-43f9-b995-8a7f6f27f3b2-kube-api-access-82529\") pod \"913564d1-5f60-43f9-b995-8a7f6f27f3b2\" (UID: \"913564d1-5f60-43f9-b995-8a7f6f27f3b2\") "
Jun 02 04:56:09.253705 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:09.253678    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/913564d1-5f60-43f9-b995-8a7f6f27f3b2-kube-api-access-82529" (OuterVolumeSpecName: "kube-api-access-82529") pod "913564d1-5f60-43f9-b995-8a7f6f27f3b2" (UID: "913564d1-5f60-43f9-b995-8a7f6f27f3b2"). InnerVolumeSpecName "kube-api-access-82529". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 04:56:09.352850 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:09.352802    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/913564d1-5f60-43f9-b995-8a7f6f27f3b2-tls-cert\") pod \"913564d1-5f60-43f9-b995-8a7f6f27f3b2\" (UID: \"913564d1-5f60-43f9-b995-8a7f6f27f3b2\") "
Jun 02 04:56:09.353201 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:09.353164    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82529\" (UniqueName: \"kubernetes.io/projected/913564d1-5f60-43f9-b995-8a7f6f27f3b2-kube-api-access-82529\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:56:09.363245 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:09.363222    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/913564d1-5f60-43f9-b995-8a7f6f27f3b2-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "913564d1-5f60-43f9-b995-8a7f6f27f3b2" (UID: "913564d1-5f60-43f9-b995-8a7f6f27f3b2"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 04:56:09.453827 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:09.453790    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/913564d1-5f60-43f9-b995-8a7f6f27f3b2-tls-cert\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:56:09.888106 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:09.888070    2577 generic.go:358] "Generic (PLEG): container finished" podID="913564d1-5f60-43f9-b995-8a7f6f27f3b2" containerID="d288e0a3d0f968992ef77ce1de1b9dac4eb42c91482a9d38f6f57da75fabcae8" exitCode=0
Jun 02 04:56:09.888524 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:09.888118    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6c858c6f-465pp"
Jun 02 04:56:09.888524 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:09.888161    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6c858c6f-465pp" event={"ID":"913564d1-5f60-43f9-b995-8a7f6f27f3b2","Type":"ContainerDied","Data":"d288e0a3d0f968992ef77ce1de1b9dac4eb42c91482a9d38f6f57da75fabcae8"}
Jun 02 04:56:09.888524 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:09.888194    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6c858c6f-465pp" event={"ID":"913564d1-5f60-43f9-b995-8a7f6f27f3b2","Type":"ContainerDied","Data":"1fe9d0c7a1421c4995436a8fbfe69d79cadbe767254f261aecc2586d61e55745"}
Jun 02 04:56:09.888524 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:09.888213    2577 scope.go:117] "RemoveContainer" containerID="d288e0a3d0f968992ef77ce1de1b9dac4eb42c91482a9d38f6f57da75fabcae8"
Jun 02 04:56:09.896485 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:09.896469    2577 scope.go:117] "RemoveContainer" containerID="d288e0a3d0f968992ef77ce1de1b9dac4eb42c91482a9d38f6f57da75fabcae8"
Jun 02 04:56:09.896715 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:56:09.896698    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d288e0a3d0f968992ef77ce1de1b9dac4eb42c91482a9d38f6f57da75fabcae8\": container with ID starting with d288e0a3d0f968992ef77ce1de1b9dac4eb42c91482a9d38f6f57da75fabcae8 not found: ID does not exist" containerID="d288e0a3d0f968992ef77ce1de1b9dac4eb42c91482a9d38f6f57da75fabcae8"
Jun 02 04:56:09.896778 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:09.896722    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d288e0a3d0f968992ef77ce1de1b9dac4eb42c91482a9d38f6f57da75fabcae8"} err="failed to get container status \"d288e0a3d0f968992ef77ce1de1b9dac4eb42c91482a9d38f6f57da75fabcae8\": rpc error: code = NotFound desc = could not find container \"d288e0a3d0f968992ef77ce1de1b9dac4eb42c91482a9d38f6f57da75fabcae8\": container with ID starting with d288e0a3d0f968992ef77ce1de1b9dac4eb42c91482a9d38f6f57da75fabcae8 not found: ID does not exist"
Jun 02 04:56:09.906178 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:09.906156    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b6c858c6f-465pp"]
Jun 02 04:56:09.909838 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:09.909794    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b6c858c6f-465pp"]
Jun 02 04:56:11.444147 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:11.444116    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="913564d1-5f60-43f9-b995-8a7f6f27f3b2" path="/var/lib/kubelet/pods/913564d1-5f60-43f9-b995-8a7f6f27f3b2/volumes"
Jun 02 04:56:19.250787 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.250749    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"]
Jun 02 04:56:19.251801 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.251778    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="913564d1-5f60-43f9-b995-8a7f6f27f3b2" containerName="authorino"
Jun 02 04:56:19.251801 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.251801    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="913564d1-5f60-43f9-b995-8a7f6f27f3b2" containerName="authorino"
Jun 02 04:56:19.251973 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.251889    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="913564d1-5f60-43f9-b995-8a7f6f27f3b2" containerName="authorino"
Jun 02 04:56:19.256230 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.256212    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.258906 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.258872    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jun 02 04:56:19.258906 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.258889    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2hdf7\""
Jun 02 04:56:19.258906 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.258896    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jun 02 04:56:19.259127 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.259023    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jun 02 04:56:19.262460 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.262439    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"]
Jun 02 04:56:19.328120 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.328091    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99ace022-73e9-4fb2-8d57-7b388c67694d-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.328239 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.328125    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99ace022-73e9-4fb2-8d57-7b388c67694d-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.328239 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.328180    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhkrk\" (UniqueName: \"kubernetes.io/projected/99ace022-73e9-4fb2-8d57-7b388c67694d-kube-api-access-bhkrk\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.328325 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.328244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99ace022-73e9-4fb2-8d57-7b388c67694d-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.328325 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.328292    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99ace022-73e9-4fb2-8d57-7b388c67694d-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.328325 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.328319    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/99ace022-73e9-4fb2-8d57-7b388c67694d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.429147 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.429116    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bhkrk\" (UniqueName: \"kubernetes.io/projected/99ace022-73e9-4fb2-8d57-7b388c67694d-kube-api-access-bhkrk\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.429295 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.429166    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99ace022-73e9-4fb2-8d57-7b388c67694d-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.429295 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.429210    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99ace022-73e9-4fb2-8d57-7b388c67694d-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.429295 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.429245    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/99ace022-73e9-4fb2-8d57-7b388c67694d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.429447 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.429312    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99ace022-73e9-4fb2-8d57-7b388c67694d-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.429447 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.429347    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99ace022-73e9-4fb2-8d57-7b388c67694d-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.429628 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.429607    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99ace022-73e9-4fb2-8d57-7b388c67694d-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.429705 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.429651    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/99ace022-73e9-4fb2-8d57-7b388c67694d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.429761 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.429717    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99ace022-73e9-4fb2-8d57-7b388c67694d-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.431722 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.431701    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99ace022-73e9-4fb2-8d57-7b388c67694d-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.432258 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.432235    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99ace022-73e9-4fb2-8d57-7b388c67694d-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.438660 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.438634    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhkrk\" (UniqueName: \"kubernetes.io/projected/99ace022-73e9-4fb2-8d57-7b388c67694d-kube-api-access-bhkrk\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc\" (UID: \"99ace022-73e9-4fb2-8d57-7b388c67694d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.567648 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.567572    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:19.699876 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.699853    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"]
Jun 02 04:56:19.701689 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:56:19.701663    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99ace022_73e9_4fb2_8d57_7b388c67694d.slice/crio-525bc09ccdc6811cd56e58634729030b124d4877a11f7666f2d23557ae3798b6 WatchSource:0}: Error finding container 525bc09ccdc6811cd56e58634729030b124d4877a11f7666f2d23557ae3798b6: Status 404 returned error can't find the container with id 525bc09ccdc6811cd56e58634729030b124d4877a11f7666f2d23557ae3798b6
Jun 02 04:56:19.926869 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:19.926773    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc" event={"ID":"99ace022-73e9-4fb2-8d57-7b388c67694d","Type":"ContainerStarted","Data":"525bc09ccdc6811cd56e58634729030b124d4877a11f7666f2d23557ae3798b6"}
Jun 02 04:56:22.940097 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:22.940063    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc" event={"ID":"99ace022-73e9-4fb2-8d57-7b388c67694d","Type":"ContainerStarted","Data":"e4f32df597556be536698c4696f82c435f79a0d028d1371a3526276a124413d5"}
Jun 02 04:56:23.944738 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:23.944708    2577 generic.go:358] "Generic (PLEG): container finished" podID="99ace022-73e9-4fb2-8d57-7b388c67694d" containerID="e4f32df597556be536698c4696f82c435f79a0d028d1371a3526276a124413d5" exitCode=0
Jun 02 04:56:23.945053 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:23.944744    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc" event={"ID":"99ace022-73e9-4fb2-8d57-7b388c67694d","Type":"ContainerDied","Data":"e4f32df597556be536698c4696f82c435f79a0d028d1371a3526276a124413d5"}
Jun 02 04:56:25.954180 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:25.954143    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc" event={"ID":"99ace022-73e9-4fb2-8d57-7b388c67694d","Type":"ContainerStarted","Data":"f40044209712f3952552e24cfd67027a24a466514b5c7a6fb9235d19bbddebe4"}
Jun 02 04:56:25.977765 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:25.977713    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc" podStartSLOduration=1.730166259 podStartE2EDuration="6.977700008s" podCreationTimestamp="2026-06-02 04:56:19 +0000 UTC" firstStartedPulling="2026-06-02 04:56:19.703501308 +0000 UTC m=+826.744432257" lastFinishedPulling="2026-06-02 04:56:24.951035057 +0000 UTC m=+831.991966006" observedRunningTime="2026-06-02 04:56:25.974856495 +0000 UTC m=+833.015787470" watchObservedRunningTime="2026-06-02 04:56:25.977700008 +0000 UTC m=+833.018631021"
Jun 02 04:56:29.568645 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:29.568594    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:29.568645 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:29.568655    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:29.581846 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:29.581806    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:29.981019 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:29.980936    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc"
Jun 02 04:56:44.157836 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.157772    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"]
Jun 02 04:56:44.163132 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.163109    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.166240 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.166214    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jun 02 04:56:44.170651 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.170251    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"]
Jun 02 04:56:44.242720 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.242693    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.242863 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.242730    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.242863 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.242784    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.242863 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.242804    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.242863 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.242852    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25kv7\" (UniqueName: \"kubernetes.io/projected/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-kube-api-access-25kv7\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.243025 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.242885    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.344270 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.344239    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.344391 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.344274    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.344391 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.344296    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.344465 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.344414    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.344465 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.344455    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-25kv7\" (UniqueName: \"kubernetes.io/projected/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-kube-api-access-25kv7\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.344545 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.344503    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.344866 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.344833    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.344973 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.344843    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.345023 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.344970    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.346755 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.346732    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.346884 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.346869    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.352405 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.352386    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25kv7\" (UniqueName: \"kubernetes.io/projected/e235cf42-bb39-4ee7-a07b-2449b67b6fd1-kube-api-access-25kv7\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc\" (UID: \"e235cf42-bb39-4ee7-a07b-2449b67b6fd1\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.474486 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.474399    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:44.603995 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:44.603970    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"]
Jun 02 04:56:44.606105 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:56:44.606080    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode235cf42_bb39_4ee7_a07b_2449b67b6fd1.slice/crio-4d1bf790a8caf1c67ff4926d5309372e51b397e8df080ca77eb9160a9ce3157d WatchSource:0}: Error finding container 4d1bf790a8caf1c67ff4926d5309372e51b397e8df080ca77eb9160a9ce3157d: Status 404 returned error can't find the container with id 4d1bf790a8caf1c67ff4926d5309372e51b397e8df080ca77eb9160a9ce3157d
Jun 02 04:56:45.027152 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:45.027118    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc" event={"ID":"e235cf42-bb39-4ee7-a07b-2449b67b6fd1","Type":"ContainerStarted","Data":"11fe06a91a1d5896006fc0f08555b9b510efde4a948e6f02b37f2f18900a3657"}
Jun 02 04:56:45.027152 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:45.027155    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc" event={"ID":"e235cf42-bb39-4ee7-a07b-2449b67b6fd1","Type":"ContainerStarted","Data":"4d1bf790a8caf1c67ff4926d5309372e51b397e8df080ca77eb9160a9ce3157d"}
Jun 02 04:56:46.032049 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:46.032009    2577 generic.go:358] "Generic (PLEG): container finished" podID="e235cf42-bb39-4ee7-a07b-2449b67b6fd1" containerID="11fe06a91a1d5896006fc0f08555b9b510efde4a948e6f02b37f2f18900a3657" exitCode=0
Jun 02 04:56:46.032487 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:46.032092    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc" event={"ID":"e235cf42-bb39-4ee7-a07b-2449b67b6fd1","Type":"ContainerDied","Data":"11fe06a91a1d5896006fc0f08555b9b510efde4a948e6f02b37f2f18900a3657"}
Jun 02 04:56:47.037465 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:47.037436    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc" event={"ID":"e235cf42-bb39-4ee7-a07b-2449b67b6fd1","Type":"ContainerStarted","Data":"a55fd91c9cc4dc7161d844f8322e87a9cfd7569175b726557c21f94dd3612a44"}
Jun 02 04:56:47.058492 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:47.058438    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc" podStartSLOduration=2.8327096259999998 podStartE2EDuration="3.058421532s" podCreationTimestamp="2026-06-02 04:56:44 +0000 UTC" firstStartedPulling="2026-06-02 04:56:46.032609445 +0000 UTC m=+853.073540395" lastFinishedPulling="2026-06-02 04:56:46.258321347 +0000 UTC m=+853.299252301" observedRunningTime="2026-06-02 04:56:47.058010729 +0000 UTC m=+854.098941702" watchObservedRunningTime="2026-06-02 04:56:47.058421532 +0000 UTC m=+854.099352505"
Jun 02 04:56:54.475006 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:54.474972    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:54.475006 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:54.475014    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:54.487665 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:54.487642    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:56:55.077300 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:56:55.077275    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc"
Jun 02 04:57:33.400244 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:33.400218    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 04:57:33.401513 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:33.401490    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 04:57:34.840154 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:34.840120    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b6f9f59f4-4jfvt"]
Jun 02 04:57:34.843695 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:34.843673    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6f9f59f4-4jfvt"
Jun 02 04:57:34.851085 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:34.851064    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b6f9f59f4-4jfvt"]
Jun 02 04:57:34.966496 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:34.966469    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cb9v\" (UniqueName: \"kubernetes.io/projected/b7740e72-00a6-467f-9e52-5507cb262118-kube-api-access-4cb9v\") pod \"authorino-b6f9f59f4-4jfvt\" (UID: \"b7740e72-00a6-467f-9e52-5507cb262118\") " pod="kuadrant-system/authorino-b6f9f59f4-4jfvt"
Jun 02 04:57:34.966640 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:34.966525    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b7740e72-00a6-467f-9e52-5507cb262118-tls-cert\") pod \"authorino-b6f9f59f4-4jfvt\" (UID: \"b7740e72-00a6-467f-9e52-5507cb262118\") " pod="kuadrant-system/authorino-b6f9f59f4-4jfvt"
Jun 02 04:57:34.966640 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:34.966606    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/b7740e72-00a6-467f-9e52-5507cb262118-oidc-ca\") pod \"authorino-b6f9f59f4-4jfvt\" (UID: \"b7740e72-00a6-467f-9e52-5507cb262118\") " pod="kuadrant-system/authorino-b6f9f59f4-4jfvt"
Jun 02 04:57:35.067346 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:35.067309    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4cb9v\" (UniqueName: \"kubernetes.io/projected/b7740e72-00a6-467f-9e52-5507cb262118-kube-api-access-4cb9v\") pod \"authorino-b6f9f59f4-4jfvt\" (UID: \"b7740e72-00a6-467f-9e52-5507cb262118\") " pod="kuadrant-system/authorino-b6f9f59f4-4jfvt"
Jun 02 04:57:35.067514 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:35.067366    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b7740e72-00a6-467f-9e52-5507cb262118-tls-cert\") pod \"authorino-b6f9f59f4-4jfvt\" (UID: \"b7740e72-00a6-467f-9e52-5507cb262118\") " pod="kuadrant-system/authorino-b6f9f59f4-4jfvt"
Jun 02 04:57:35.067514 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:35.067421    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/b7740e72-00a6-467f-9e52-5507cb262118-oidc-ca\") pod \"authorino-b6f9f59f4-4jfvt\" (UID: \"b7740e72-00a6-467f-9e52-5507cb262118\") " pod="kuadrant-system/authorino-b6f9f59f4-4jfvt"
Jun 02 04:57:35.068084 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:35.068059    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/b7740e72-00a6-467f-9e52-5507cb262118-oidc-ca\") pod \"authorino-b6f9f59f4-4jfvt\" (UID: \"b7740e72-00a6-467f-9e52-5507cb262118\") " pod="kuadrant-system/authorino-b6f9f59f4-4jfvt"
Jun 02 04:57:35.069702 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:35.069680    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b7740e72-00a6-467f-9e52-5507cb262118-tls-cert\") pod \"authorino-b6f9f59f4-4jfvt\" (UID: \"b7740e72-00a6-467f-9e52-5507cb262118\") " pod="kuadrant-system/authorino-b6f9f59f4-4jfvt"
Jun 02 04:57:35.075310 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:35.075288    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cb9v\" (UniqueName: \"kubernetes.io/projected/b7740e72-00a6-467f-9e52-5507cb262118-kube-api-access-4cb9v\") pod \"authorino-b6f9f59f4-4jfvt\" (UID: \"b7740e72-00a6-467f-9e52-5507cb262118\") " pod="kuadrant-system/authorino-b6f9f59f4-4jfvt"
Jun 02 04:57:35.153998 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:35.153940    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6f9f59f4-4jfvt"
Jun 02 04:57:35.288644 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:35.288597    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b6f9f59f4-4jfvt"]
Jun 02 04:57:35.291096 ip-10-0-135-23 kubenswrapper[2577]: W0602 04:57:35.291067    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7740e72_00a6_467f_9e52_5507cb262118.slice/crio-1f605f1e6f1258d8fbbc121f0e613b3932efcff178007e12aed6d9ce04bbfcbb WatchSource:0}: Error finding container 1f605f1e6f1258d8fbbc121f0e613b3932efcff178007e12aed6d9ce04bbfcbb: Status 404 returned error can't find the container with id 1f605f1e6f1258d8fbbc121f0e613b3932efcff178007e12aed6d9ce04bbfcbb
Jun 02 04:57:36.216005 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:36.215911    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6f9f59f4-4jfvt" event={"ID":"b7740e72-00a6-467f-9e52-5507cb262118","Type":"ContainerStarted","Data":"f44219ad3a6efdd8554e7f712298a91c18904bd876313d15629cf8be2724c56e"}
Jun 02 04:57:36.216005 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:36.215959    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6f9f59f4-4jfvt" event={"ID":"b7740e72-00a6-467f-9e52-5507cb262118","Type":"ContainerStarted","Data":"1f605f1e6f1258d8fbbc121f0e613b3932efcff178007e12aed6d9ce04bbfcbb"}
Jun 02 04:57:36.236736 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:36.236673    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-b6f9f59f4-4jfvt" podStartSLOduration=1.777103163 podStartE2EDuration="2.236656867s" podCreationTimestamp="2026-06-02 04:57:34 +0000 UTC" firstStartedPulling="2026-06-02 04:57:35.292846826 +0000 UTC m=+902.333777781" lastFinishedPulling="2026-06-02 04:57:35.752400532 +0000 UTC m=+902.793331485" observedRunningTime="2026-06-02 04:57:36.235912 +0000 UTC m=+903.276843035" watchObservedRunningTime="2026-06-02 04:57:36.236656867 +0000 UTC m=+903.277587907"
Jun 02 04:57:36.270984 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:36.270326    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-669dbdf959-9btr2"]
Jun 02 04:57:36.270984 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:36.270642    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-669dbdf959-9btr2" podUID="7d30e8bd-f3ef-4480-a62d-c9602fdd92ae" containerName="authorino" containerID="cri-o://d8998d03788047acdcbee888c241ba60f664093ac449387d5fe55d9f9d962154" gracePeriod=30
Jun 02 04:57:36.525343 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:36.525322    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-669dbdf959-9btr2"
Jun 02 04:57:36.683373 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:36.683344    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-oidc-ca\") pod \"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae\" (UID: \"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae\") "
Jun 02 04:57:36.683508 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:36.683415    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhvqv\" (UniqueName: \"kubernetes.io/projected/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-kube-api-access-zhvqv\") pod \"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae\" (UID: \"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae\") "
Jun 02 04:57:36.683508 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:36.683437    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-tls-cert\") pod \"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae\" (UID: \"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae\") "
Jun 02 04:57:36.685390 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:36.685364    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-kube-api-access-zhvqv" (OuterVolumeSpecName: "kube-api-access-zhvqv") pod "7d30e8bd-f3ef-4480-a62d-c9602fdd92ae" (UID: "7d30e8bd-f3ef-4480-a62d-c9602fdd92ae"). InnerVolumeSpecName "kube-api-access-zhvqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 04:57:36.688289 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:36.688265    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "7d30e8bd-f3ef-4480-a62d-c9602fdd92ae" (UID: "7d30e8bd-f3ef-4480-a62d-c9602fdd92ae"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jun 02 04:57:36.692954 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:36.692934    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "7d30e8bd-f3ef-4480-a62d-c9602fdd92ae" (UID: "7d30e8bd-f3ef-4480-a62d-c9602fdd92ae"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jun 02 04:57:36.784535 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:36.784475    2577 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-oidc-ca\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:57:36.784535 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:36.784501    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhvqv\" (UniqueName: \"kubernetes.io/projected/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-kube-api-access-zhvqv\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:57:36.784535 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:36.784512    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae-tls-cert\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:57:37.220450 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:37.220416    2577 generic.go:358] "Generic (PLEG): container finished" podID="7d30e8bd-f3ef-4480-a62d-c9602fdd92ae" containerID="d8998d03788047acdcbee888c241ba60f664093ac449387d5fe55d9f9d962154" exitCode=0
Jun 02 04:57:37.220807 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:37.220464    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-669dbdf959-9btr2"
Jun 02 04:57:37.220807 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:37.220497    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-669dbdf959-9btr2" event={"ID":"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae","Type":"ContainerDied","Data":"d8998d03788047acdcbee888c241ba60f664093ac449387d5fe55d9f9d962154"}
Jun 02 04:57:37.220807 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:37.220534    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-669dbdf959-9btr2" event={"ID":"7d30e8bd-f3ef-4480-a62d-c9602fdd92ae","Type":"ContainerDied","Data":"fdd3964a1482441cdfcba7a2903a72d475376d19806d9665c213ae76b94638b5"}
Jun 02 04:57:37.220807 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:37.220551    2577 scope.go:117] "RemoveContainer" containerID="d8998d03788047acdcbee888c241ba60f664093ac449387d5fe55d9f9d962154"
Jun 02 04:57:37.228723 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:37.228704    2577 scope.go:117] "RemoveContainer" containerID="d8998d03788047acdcbee888c241ba60f664093ac449387d5fe55d9f9d962154"
Jun 02 04:57:37.228998 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:57:37.228981    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8998d03788047acdcbee888c241ba60f664093ac449387d5fe55d9f9d962154\": container with ID starting with d8998d03788047acdcbee888c241ba60f664093ac449387d5fe55d9f9d962154 not found: ID does not exist" containerID="d8998d03788047acdcbee888c241ba60f664093ac449387d5fe55d9f9d962154"
Jun 02 04:57:37.229063 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:37.229006    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8998d03788047acdcbee888c241ba60f664093ac449387d5fe55d9f9d962154"} err="failed to get container status \"d8998d03788047acdcbee888c241ba60f664093ac449387d5fe55d9f9d962154\": rpc error: code = NotFound desc = could not find container \"d8998d03788047acdcbee888c241ba60f664093ac449387d5fe55d9f9d962154\": container with ID starting with d8998d03788047acdcbee888c241ba60f664093ac449387d5fe55d9f9d962154 not found: ID does not exist"
Jun 02 04:57:37.240995 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:37.240975    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-669dbdf959-9btr2"]
Jun 02 04:57:37.244178 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:37.244161    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-669dbdf959-9btr2"]
Jun 02 04:57:37.444231 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:57:37.444179    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d30e8bd-f3ef-4480-a62d-c9602fdd92ae" path="/var/lib/kubelet/pods/7d30e8bd-f3ef-4480-a62d-c9602fdd92ae/volumes"
Jun 02 04:59:04.121851 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:59:04.121748    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-679b7f88b7-dzdbh"]
Jun 02 04:59:04.122652 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:59:04.122046    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-679b7f88b7-dzdbh" podUID="2b03f4b1-2cac-4a56-9d30-5cb87927c426" containerName="manager" containerID="cri-o://34ca32e9f318fd1297191c5a6e6365c49d19a26e6dd1ec9fce7eec4de7905e85" gracePeriod=30
Jun 02 04:59:04.375914 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:59:04.375845    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-679b7f88b7-dzdbh"
Jun 02 04:59:04.386017 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:59:04.385981    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgsgw\" (UniqueName: \"kubernetes.io/projected/2b03f4b1-2cac-4a56-9d30-5cb87927c426-kube-api-access-zgsgw\") pod \"2b03f4b1-2cac-4a56-9d30-5cb87927c426\" (UID: \"2b03f4b1-2cac-4a56-9d30-5cb87927c426\") "
Jun 02 04:59:04.388210 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:59:04.388174    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b03f4b1-2cac-4a56-9d30-5cb87927c426-kube-api-access-zgsgw" (OuterVolumeSpecName: "kube-api-access-zgsgw") pod "2b03f4b1-2cac-4a56-9d30-5cb87927c426" (UID: "2b03f4b1-2cac-4a56-9d30-5cb87927c426"). InnerVolumeSpecName "kube-api-access-zgsgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 04:59:04.487254 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:59:04.487223    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgsgw\" (UniqueName: \"kubernetes.io/projected/2b03f4b1-2cac-4a56-9d30-5cb87927c426-kube-api-access-zgsgw\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 04:59:04.526763 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:59:04.526738    2577 generic.go:358] "Generic (PLEG): container finished" podID="2b03f4b1-2cac-4a56-9d30-5cb87927c426" containerID="34ca32e9f318fd1297191c5a6e6365c49d19a26e6dd1ec9fce7eec4de7905e85" exitCode=0
Jun 02 04:59:04.526903 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:59:04.526779    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-679b7f88b7-dzdbh" event={"ID":"2b03f4b1-2cac-4a56-9d30-5cb87927c426","Type":"ContainerDied","Data":"34ca32e9f318fd1297191c5a6e6365c49d19a26e6dd1ec9fce7eec4de7905e85"}
Jun 02 04:59:04.526903 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:59:04.526783    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-679b7f88b7-dzdbh"
Jun 02 04:59:04.526903 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:59:04.526799    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-679b7f88b7-dzdbh" event={"ID":"2b03f4b1-2cac-4a56-9d30-5cb87927c426","Type":"ContainerDied","Data":"3a5d14dd301d1170d5bbfd6a8afcfaf22defbb9af14080a51297026f78e52562"}
Jun 02 04:59:04.526903 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:59:04.526837    2577 scope.go:117] "RemoveContainer" containerID="34ca32e9f318fd1297191c5a6e6365c49d19a26e6dd1ec9fce7eec4de7905e85"
Jun 02 04:59:04.535076 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:59:04.535051    2577 scope.go:117] "RemoveContainer" containerID="34ca32e9f318fd1297191c5a6e6365c49d19a26e6dd1ec9fce7eec4de7905e85"
Jun 02 04:59:04.535327 ip-10-0-135-23 kubenswrapper[2577]: E0602 04:59:04.535307    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34ca32e9f318fd1297191c5a6e6365c49d19a26e6dd1ec9fce7eec4de7905e85\": container with ID starting with 34ca32e9f318fd1297191c5a6e6365c49d19a26e6dd1ec9fce7eec4de7905e85 not found: ID does not exist" containerID="34ca32e9f318fd1297191c5a6e6365c49d19a26e6dd1ec9fce7eec4de7905e85"
Jun 02 04:59:04.535379 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:59:04.535333    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34ca32e9f318fd1297191c5a6e6365c49d19a26e6dd1ec9fce7eec4de7905e85"} err="failed to get container status \"34ca32e9f318fd1297191c5a6e6365c49d19a26e6dd1ec9fce7eec4de7905e85\": rpc error: code = NotFound desc = could not find container \"34ca32e9f318fd1297191c5a6e6365c49d19a26e6dd1ec9fce7eec4de7905e85\": container with ID starting with 34ca32e9f318fd1297191c5a6e6365c49d19a26e6dd1ec9fce7eec4de7905e85 not found: ID does not exist"
Jun 02 04:59:04.548252 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:59:04.548227    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-679b7f88b7-dzdbh"]
Jun 02 04:59:04.551749 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:59:04.551729    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-679b7f88b7-dzdbh"]
Jun 02 04:59:05.445092 ip-10-0-135-23 kubenswrapper[2577]: I0602 04:59:05.445060    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b03f4b1-2cac-4a56-9d30-5cb87927c426" path="/var/lib/kubelet/pods/2b03f4b1-2cac-4a56-9d30-5cb87927c426/volumes"
Jun 02 05:00:00.210509 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.210477    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29672940-p2cx4"]
Jun 02 05:00:00.210930 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.210839    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b03f4b1-2cac-4a56-9d30-5cb87927c426" containerName="manager"
Jun 02 05:00:00.210930 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.210854    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b03f4b1-2cac-4a56-9d30-5cb87927c426" containerName="manager"
Jun 02 05:00:00.210930 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.210871    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d30e8bd-f3ef-4480-a62d-c9602fdd92ae" containerName="authorino"
Jun 02 05:00:00.210930 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.210878    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d30e8bd-f3ef-4480-a62d-c9602fdd92ae" containerName="authorino"
Jun 02 05:00:00.211061 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.210948    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="7d30e8bd-f3ef-4480-a62d-c9602fdd92ae" containerName="authorino"
Jun 02 05:00:00.211061 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.210960    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="2b03f4b1-2cac-4a56-9d30-5cb87927c426" containerName="manager"
Jun 02 05:00:00.213974 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.213959    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672940-p2cx4"
Jun 02 05:00:00.216628 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.216606    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-s98rm\""
Jun 02 05:00:00.220784 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.220763    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672940-p2cx4"]
Jun 02 05:00:00.338119 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.338081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj7xj\" (UniqueName: \"kubernetes.io/projected/aaa45c49-175d-40e0-b626-173babd8f3b1-kube-api-access-zj7xj\") pod \"maas-api-key-cleanup-29672940-p2cx4\" (UID: \"aaa45c49-175d-40e0-b626-173babd8f3b1\") " pod="opendatahub/maas-api-key-cleanup-29672940-p2cx4"
Jun 02 05:00:00.438899 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.438863    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zj7xj\" (UniqueName: \"kubernetes.io/projected/aaa45c49-175d-40e0-b626-173babd8f3b1-kube-api-access-zj7xj\") pod \"maas-api-key-cleanup-29672940-p2cx4\" (UID: \"aaa45c49-175d-40e0-b626-173babd8f3b1\") " pod="opendatahub/maas-api-key-cleanup-29672940-p2cx4"
Jun 02 05:00:00.449563 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.449533    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj7xj\" (UniqueName: \"kubernetes.io/projected/aaa45c49-175d-40e0-b626-173babd8f3b1-kube-api-access-zj7xj\") pod \"maas-api-key-cleanup-29672940-p2cx4\" (UID: \"aaa45c49-175d-40e0-b626-173babd8f3b1\") " pod="opendatahub/maas-api-key-cleanup-29672940-p2cx4"
Jun 02 05:00:00.526088 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.526067    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672940-p2cx4"
Jun 02 05:00:00.656295 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.656265    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672940-p2cx4"]
Jun 02 05:00:00.659152 ip-10-0-135-23 kubenswrapper[2577]: W0602 05:00:00.659122    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa45c49_175d_40e0_b626_173babd8f3b1.slice/crio-919d969c9812a621c7800482067c685b97689063e30266a49684a72d99f16e21 WatchSource:0}: Error finding container 919d969c9812a621c7800482067c685b97689063e30266a49684a72d99f16e21: Status 404 returned error can't find the container with id 919d969c9812a621c7800482067c685b97689063e30266a49684a72d99f16e21
Jun 02 05:00:00.660868 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.660851    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 05:00:00.722989 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:00.722955    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672940-p2cx4" event={"ID":"aaa45c49-175d-40e0-b626-173babd8f3b1","Type":"ContainerStarted","Data":"919d969c9812a621c7800482067c685b97689063e30266a49684a72d99f16e21"}
Jun 02 05:00:03.734648 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:03.734620    2577 generic.go:358] "Generic (PLEG): container finished" podID="aaa45c49-175d-40e0-b626-173babd8f3b1" containerID="fa64ea8a296385e6055975c98ae2caaeec63758c9264ec074457409b22bfa517" exitCode=0
Jun 02 05:00:03.734996 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:03.734657    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672940-p2cx4" event={"ID":"aaa45c49-175d-40e0-b626-173babd8f3b1","Type":"ContainerDied","Data":"fa64ea8a296385e6055975c98ae2caaeec63758c9264ec074457409b22bfa517"}
Jun 02 05:00:04.866061 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:04.866040    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672940-p2cx4"
Jun 02 05:00:04.978038 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:04.978012    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zj7xj\" (UniqueName: \"kubernetes.io/projected/aaa45c49-175d-40e0-b626-173babd8f3b1-kube-api-access-zj7xj\") pod \"aaa45c49-175d-40e0-b626-173babd8f3b1\" (UID: \"aaa45c49-175d-40e0-b626-173babd8f3b1\") "
Jun 02 05:00:04.979805 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:04.979781    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaa45c49-175d-40e0-b626-173babd8f3b1-kube-api-access-zj7xj" (OuterVolumeSpecName: "kube-api-access-zj7xj") pod "aaa45c49-175d-40e0-b626-173babd8f3b1" (UID: "aaa45c49-175d-40e0-b626-173babd8f3b1"). InnerVolumeSpecName "kube-api-access-zj7xj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 05:00:05.079432 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:05.079374    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zj7xj\" (UniqueName: \"kubernetes.io/projected/aaa45c49-175d-40e0-b626-173babd8f3b1-kube-api-access-zj7xj\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 05:00:05.742323 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:05.742289    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672940-p2cx4" event={"ID":"aaa45c49-175d-40e0-b626-173babd8f3b1","Type":"ContainerDied","Data":"919d969c9812a621c7800482067c685b97689063e30266a49684a72d99f16e21"}
Jun 02 05:00:05.742468 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:05.742329    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="919d969c9812a621c7800482067c685b97689063e30266a49684a72d99f16e21"
Jun 02 05:00:05.742468 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:00:05.742360    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672940-p2cx4"
Jun 02 05:02:33.429947 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:02:33.429917    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 05:02:33.431016 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:02:33.430996    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 05:07:33.459402 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:07:33.459374    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 05:07:33.461854 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:07:33.461552    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 05:12:33.484086 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:12:33.483982    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 05:12:33.485666 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:12:33.485648    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 05:15:00.136700 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:00.136658    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29672955-x2qhg"]
Jun 02 05:15:00.137300 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:00.137227    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaa45c49-175d-40e0-b626-173babd8f3b1" containerName="cleanup"
Jun 02 05:15:00.137300 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:00.137251    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaa45c49-175d-40e0-b626-173babd8f3b1" containerName="cleanup"
Jun 02 05:15:00.137419 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:00.137333    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="aaa45c49-175d-40e0-b626-173babd8f3b1" containerName="cleanup"
Jun 02 05:15:00.140439 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:00.140417    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672955-x2qhg"
Jun 02 05:15:00.142943 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:00.142925    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-s98rm\""
Jun 02 05:15:00.155135 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:00.155108    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672955-x2qhg"]
Jun 02 05:15:00.245667 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:00.245644    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f698z\" (UniqueName: \"kubernetes.io/projected/5a942609-06c9-4c69-b1f4-b5efe33cd7d7-kube-api-access-f698z\") pod \"maas-api-key-cleanup-29672955-x2qhg\" (UID: \"5a942609-06c9-4c69-b1f4-b5efe33cd7d7\") " pod="opendatahub/maas-api-key-cleanup-29672955-x2qhg"
Jun 02 05:15:00.347088 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:00.347062    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f698z\" (UniqueName: \"kubernetes.io/projected/5a942609-06c9-4c69-b1f4-b5efe33cd7d7-kube-api-access-f698z\") pod \"maas-api-key-cleanup-29672955-x2qhg\" (UID: \"5a942609-06c9-4c69-b1f4-b5efe33cd7d7\") " pod="opendatahub/maas-api-key-cleanup-29672955-x2qhg"
Jun 02 05:15:00.355545 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:00.355521    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f698z\" (UniqueName: \"kubernetes.io/projected/5a942609-06c9-4c69-b1f4-b5efe33cd7d7-kube-api-access-f698z\") pod \"maas-api-key-cleanup-29672955-x2qhg\" (UID: \"5a942609-06c9-4c69-b1f4-b5efe33cd7d7\") " pod="opendatahub/maas-api-key-cleanup-29672955-x2qhg"
Jun 02 05:15:00.450137 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:00.450089    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672955-x2qhg"
Jun 02 05:15:00.574428 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:00.574406    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672955-x2qhg"]
Jun 02 05:15:00.576990 ip-10-0-135-23 kubenswrapper[2577]: W0602 05:15:00.576965    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a942609_06c9_4c69_b1f4_b5efe33cd7d7.slice/crio-91c25281f4347fa0ad94d9e3f60c6c05ce393092c070825d8a8e561c2411148f WatchSource:0}: Error finding container 91c25281f4347fa0ad94d9e3f60c6c05ce393092c070825d8a8e561c2411148f: Status 404 returned error can't find the container with id 91c25281f4347fa0ad94d9e3f60c6c05ce393092c070825d8a8e561c2411148f
Jun 02 05:15:00.579202 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:00.579187    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 05:15:00.813143 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:00.813119    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672955-x2qhg" event={"ID":"5a942609-06c9-4c69-b1f4-b5efe33cd7d7","Type":"ContainerStarted","Data":"91c25281f4347fa0ad94d9e3f60c6c05ce393092c070825d8a8e561c2411148f"}
Jun 02 05:15:01.817860 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:01.817834    2577 generic.go:358] "Generic (PLEG): container finished" podID="5a942609-06c9-4c69-b1f4-b5efe33cd7d7" containerID="d4a98c11ae62286102cb0791be714ffda356b8f0950584b8d9b1cab321f725ce" exitCode=0
Jun 02 05:15:01.818166 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:01.817923    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672955-x2qhg" event={"ID":"5a942609-06c9-4c69-b1f4-b5efe33cd7d7","Type":"ContainerDied","Data":"d4a98c11ae62286102cb0791be714ffda356b8f0950584b8d9b1cab321f725ce"}
Jun 02 05:15:02.950153 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:02.950131    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672955-x2qhg"
Jun 02 05:15:03.070921 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:03.070896    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f698z\" (UniqueName: \"kubernetes.io/projected/5a942609-06c9-4c69-b1f4-b5efe33cd7d7-kube-api-access-f698z\") pod \"5a942609-06c9-4c69-b1f4-b5efe33cd7d7\" (UID: \"5a942609-06c9-4c69-b1f4-b5efe33cd7d7\") "
Jun 02 05:15:03.072875 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:03.072822    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a942609-06c9-4c69-b1f4-b5efe33cd7d7-kube-api-access-f698z" (OuterVolumeSpecName: "kube-api-access-f698z") pod "5a942609-06c9-4c69-b1f4-b5efe33cd7d7" (UID: "5a942609-06c9-4c69-b1f4-b5efe33cd7d7"). InnerVolumeSpecName "kube-api-access-f698z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jun 02 05:15:03.171775 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:03.171752    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f698z\" (UniqueName: \"kubernetes.io/projected/5a942609-06c9-4c69-b1f4-b5efe33cd7d7-kube-api-access-f698z\") on node \"ip-10-0-135-23.ec2.internal\" DevicePath \"\""
Jun 02 05:15:03.825794 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:03.825767    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672955-x2qhg" event={"ID":"5a942609-06c9-4c69-b1f4-b5efe33cd7d7","Type":"ContainerDied","Data":"91c25281f4347fa0ad94d9e3f60c6c05ce393092c070825d8a8e561c2411148f"}
Jun 02 05:15:03.825794 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:03.825796    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91c25281f4347fa0ad94d9e3f60c6c05ce393092c070825d8a8e561c2411148f"
Jun 02 05:15:03.825955 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:15:03.825838    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672955-x2qhg"
Jun 02 05:17:33.506025 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:17:33.506000    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 05:17:33.509328 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:17:33.509310    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 05:20:41.908226 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:41.908194    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b6f9f59f4-4jfvt_b7740e72-00a6-467f-9e52-5507cb262118/authorino/0.log"
Jun 02 05:20:46.551833 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:46.551783    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-5bb69db85-jblwh_3ac5a2ae-2a3b-4463-a5b5-9f091d343132/nginx/0.log"
Jun 02 05:20:46.660156 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:46.660131    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-7c99ffcdcd-4wxxn_017432b7-cfc9-4dab-8dae-4cd4b48c0479/manager/0.log"
Jun 02 05:20:46.798746 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:46.798722    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-c6b9847d7-fstgz_ae936840-9758-4f32-9197-33c2bf5e7fea/manager/0.log"
Jun 02 05:20:47.058840 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:47.058808    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29672940-p2cx4_aaa45c49-175d-40e0-b626-173babd8f3b1/cleanup/0.log"
Jun 02 05:20:47.173671 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:47.173649    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29672955-x2qhg_5a942609-06c9-4c69-b1f4-b5efe33cd7d7/cleanup/0.log"
Jun 02 05:20:47.408711 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:47.408634    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-6577596cd5-77thv_e007367e-46a0-4676-b34c-b86649eae5ff/server/0.log"
Jun 02 05:20:47.518756 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:47.518727    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-lz5df_3027d5df-6b2f-4eba-8261-5a2f6fb4e98e/manager/0.log"
Jun 02 05:20:47.876174 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:47.876150    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55fcdd494f-rww7z_2e3270e6-5b75-4617-9e28-2e27b883c151/manager/0.log"
Jun 02 05:20:47.884383 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:47.884367    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55fcdd494f-rww7z_2e3270e6-5b75-4617-9e28-2e27b883c151/copy-manifests/0.log"
Jun 02 05:20:47.998763 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:47.998743    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-sng54_ba819424-07f5-4215-98d7-534d6c678735/postgres/0.log"
Jun 02 05:20:49.279932 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:49.279903    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b6f9f59f4-4jfvt_b7740e72-00a6-467f-9e52-5507cb262118/authorino/0.log"
Jun 02 05:20:49.527623 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:49.527599    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-twmzt_df47cc68-409f-453a-b88a-cb8b6b3929b3/manager/0.log"
Jun 02 05:20:49.749179 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:49.749159    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-bsqqk_756f6c87-a8ca-4cff-a805-a3377dd45f14/registry-server/0.log"
Jun 02 05:20:50.118125 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:50.118062    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-hrbx7_82ae05cd-8b0d-44f0-8b44-cbcaf39770ca/manager/0.log"
Jun 02 05:20:50.472963 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:50.472905    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4b5gv92_158b6c5f-4896-44f2-aeba-58fcaf5a7da1/istio-proxy/0.log"
Jun 02 05:20:50.824900 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:50.820041    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77644847-zjcv7_7e88f06c-6f90-4b0d-99f0-63b5ec893ed6/kube-auth-proxy/0.log"
Jun 02 05:20:50.939480 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:50.939454    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-wvxbc_71c1c48e-df32-44e5-b88a-49bdb9380172/istio-proxy/0.log"
Jun 02 05:20:51.632843 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:51.632801    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc_99ace022-73e9-4fb2-8d57-7b388c67694d/storage-initializer/0.log"
Jun 02 05:20:51.640111 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:51.640087    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-dqsvc_99ace022-73e9-4fb2-8d57-7b388c67694d/main/0.log"
Jun 02 05:20:51.994175 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:51.994129    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc_e235cf42-bb39-4ee7-a07b-2449b67b6fd1/storage-initializer/0.log"
Jun 02 05:20:52.000732 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:52.000711    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-cfccc_e235cf42-bb39-4ee7-a07b-2449b67b6fd1/main/0.log"
Jun 02 05:20:59.807504 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:59.807472    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hhdkx_c861fa7b-2da0-4681-99b9-9d73c9469d51/global-pull-secret-syncer/0.log"
Jun 02 05:20:59.919347 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:59.919313    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qdfbx_f4ba583d-27eb-4e01-abf6-42dd3767e562/konnectivity-agent/0.log"
Jun 02 05:20:59.984996 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:20:59.984960    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-23.ec2.internal_8d89a7cb71da8618cf393c332af1d5c1/haproxy/0.log"
Jun 02 05:21:04.241194 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:04.241157    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b6f9f59f4-4jfvt_b7740e72-00a6-467f-9e52-5507cb262118/authorino/0.log"
Jun 02 05:21:04.298610 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:04.298587    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-twmzt_df47cc68-409f-453a-b88a-cb8b6b3929b3/manager/0.log"
Jun 02 05:21:04.364040 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:04.364011    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-bsqqk_756f6c87-a8ca-4cff-a805-a3377dd45f14/registry-server/0.log"
Jun 02 05:21:04.515033 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:04.514976    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-hrbx7_82ae05cd-8b0d-44f0-8b44-cbcaf39770ca/manager/0.log"
Jun 02 05:21:06.042269 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:06.042239    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-8578879b87-5w6pf_bdf23091-58b3-4d72-aa45-3659a2c2e924/metrics-server/0.log"
Jun 02 05:21:06.247081 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:06.247058    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tcvt9_9600a15e-a21e-4e02-a697-4aae0cfe34e1/node-exporter/0.log"
Jun 02 05:21:06.270220 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:06.270196    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tcvt9_9600a15e-a21e-4e02-a697-4aae0cfe34e1/kube-rbac-proxy/0.log"
Jun 02 05:21:06.291649 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:06.291634    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tcvt9_9600a15e-a21e-4e02-a697-4aae0cfe34e1/init-textfile/0.log"
Jun 02 05:21:09.193664 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.193636    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-69b7b4b6dc-r6vxr_0ef79026-ad8d-49db-a5e0-d075fe7af24f/console/0.log"
Jun 02 05:21:09.740904 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.740877    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"]
Jun 02 05:21:09.741240 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.741228    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a942609-06c9-4c69-b1f4-b5efe33cd7d7" containerName="cleanup"
Jun 02 05:21:09.741290 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.741242    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a942609-06c9-4c69-b1f4-b5efe33cd7d7" containerName="cleanup"
Jun 02 05:21:09.741335 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.741324    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="5a942609-06c9-4c69-b1f4-b5efe33cd7d7" containerName="cleanup"
Jun 02 05:21:09.744231 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.744209    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:09.746869 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.746845    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mf6h5\"/\"openshift-service-ca.crt\""
Jun 02 05:21:09.747656 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.747638    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mf6h5\"/\"default-dockercfg-s222j\""
Jun 02 05:21:09.747719 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.747657    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mf6h5\"/\"kube-root-ca.crt\""
Jun 02 05:21:09.752777 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.752755    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"]
Jun 02 05:21:09.844362 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.844340    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr5pm\" (UniqueName: \"kubernetes.io/projected/f8b838ab-68c0-44d3-913c-cd45c9ad6f0e-kube-api-access-dr5pm\") pod \"perf-node-gather-daemonset-s2m84\" (UID: \"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e\") " pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:09.844462 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.844375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8b838ab-68c0-44d3-913c-cd45c9ad6f0e-sys\") pod \"perf-node-gather-daemonset-s2m84\" (UID: \"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e\") " pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:09.844462 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.844412    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8b838ab-68c0-44d3-913c-cd45c9ad6f0e-lib-modules\") pod \"perf-node-gather-daemonset-s2m84\" (UID: \"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e\") " pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:09.844462 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.844441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f8b838ab-68c0-44d3-913c-cd45c9ad6f0e-podres\") pod \"perf-node-gather-daemonset-s2m84\" (UID: \"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e\") " pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:09.844580 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.844463    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f8b838ab-68c0-44d3-913c-cd45c9ad6f0e-proc\") pod \"perf-node-gather-daemonset-s2m84\" (UID: \"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e\") " pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:09.945684 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.945659    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f8b838ab-68c0-44d3-913c-cd45c9ad6f0e-proc\") pod \"perf-node-gather-daemonset-s2m84\" (UID: \"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e\") " pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:09.945774 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.945697    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dr5pm\" (UniqueName: \"kubernetes.io/projected/f8b838ab-68c0-44d3-913c-cd45c9ad6f0e-kube-api-access-dr5pm\") pod \"perf-node-gather-daemonset-s2m84\" (UID: \"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e\") " pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:09.945774 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.945726    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8b838ab-68c0-44d3-913c-cd45c9ad6f0e-sys\") pod \"perf-node-gather-daemonset-s2m84\" (UID: \"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e\") " pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:09.945915 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.945773    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8b838ab-68c0-44d3-913c-cd45c9ad6f0e-lib-modules\") pod \"perf-node-gather-daemonset-s2m84\" (UID: \"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e\") " pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:09.945915 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.945784    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f8b838ab-68c0-44d3-913c-cd45c9ad6f0e-proc\") pod \"perf-node-gather-daemonset-s2m84\" (UID: \"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e\") " pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:09.945915 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.945874    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8b838ab-68c0-44d3-913c-cd45c9ad6f0e-sys\") pod \"perf-node-gather-daemonset-s2m84\" (UID: \"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e\") " pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:09.945915 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.945880    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f8b838ab-68c0-44d3-913c-cd45c9ad6f0e-podres\") pod \"perf-node-gather-daemonset-s2m84\" (UID: \"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e\") " pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:09.945915 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.945906    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8b838ab-68c0-44d3-913c-cd45c9ad6f0e-lib-modules\") pod \"perf-node-gather-daemonset-s2m84\" (UID: \"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e\") " pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:09.946238 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.945969    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f8b838ab-68c0-44d3-913c-cd45c9ad6f0e-podres\") pod \"perf-node-gather-daemonset-s2m84\" (UID: \"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e\") " pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:09.954376 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:09.954355    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr5pm\" (UniqueName: \"kubernetes.io/projected/f8b838ab-68c0-44d3-913c-cd45c9ad6f0e-kube-api-access-dr5pm\") pod \"perf-node-gather-daemonset-s2m84\" (UID: \"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e\") " pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:10.054641 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:10.054595    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:10.180243 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:10.180214    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"]
Jun 02 05:21:10.183903 ip-10-0-135-23 kubenswrapper[2577]: W0602 05:21:10.183874    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf8b838ab_68c0_44d3_913c_cd45c9ad6f0e.slice/crio-87d556b78ac1f94ba5e85783a052f750da17b148feb765f47a3facd0d3a40a2c WatchSource:0}: Error finding container 87d556b78ac1f94ba5e85783a052f750da17b148feb765f47a3facd0d3a40a2c: Status 404 returned error can't find the container with id 87d556b78ac1f94ba5e85783a052f750da17b148feb765f47a3facd0d3a40a2c
Jun 02 05:21:10.185443 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:10.185426    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jun 02 05:21:10.743619 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:10.743589    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zlpr5_5a1225d4-452b-41f9-9890-2e039b99c9d6/dns/0.log"
Jun 02 05:21:10.761663 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:10.761640    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zlpr5_5a1225d4-452b-41f9-9890-2e039b99c9d6/kube-rbac-proxy/0.log"
Jun 02 05:21:10.828044 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:10.828018    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vfq72_f99eac06-93b0-4b1b-aaa0-285d781e5412/dns-node-resolver/0.log"
Jun 02 05:21:11.111430 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:11.111363    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84" event={"ID":"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e","Type":"ContainerStarted","Data":"e6354c13351d81ac0e79f6bdceb6a350163232f42e078419d9a04f79d9646b2f"}
Jun 02 05:21:11.111430 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:11.111394    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84" event={"ID":"f8b838ab-68c0-44d3-913c-cd45c9ad6f0e","Type":"ContainerStarted","Data":"87d556b78ac1f94ba5e85783a052f750da17b148feb765f47a3facd0d3a40a2c"}
Jun 02 05:21:11.111574 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:11.111451    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:11.128606 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:11.128564    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84" podStartSLOduration=2.128550765 podStartE2EDuration="2.128550765s" podCreationTimestamp="2026-06-02 05:21:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 05:21:11.127353 +0000 UTC m=+2318.168283973" watchObservedRunningTime="2026-06-02 05:21:11.128550765 +0000 UTC m=+2318.169481742"
Jun 02 05:21:11.288434 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:11.288413    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7494f5fddc-msbxk_17dfd930-9652-4845-a83a-9f40ea622fcd/registry/0.log"
Jun 02 05:21:11.306983 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:11.306964    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gk5rc_a77859ea-6f29-411d-b3d1-abc2e33e3745/node-ca/0.log"
Jun 02 05:21:12.195769 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:12.195719    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4b5gv92_158b6c5f-4896-44f2-aeba-58fcaf5a7da1/istio-proxy/0.log"
Jun 02 05:21:12.314464 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:12.314434    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77644847-zjcv7_7e88f06c-6f90-4b0d-99f0-63b5ec893ed6/kube-auth-proxy/0.log"
Jun 02 05:21:12.369830 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:12.369799    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-wvxbc_71c1c48e-df32-44e5-b88a-49bdb9380172/istio-proxy/0.log"
Jun 02 05:21:13.020323 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:13.020300    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pzswt_919222b9-64e3-4f4f-90c3-351f103d2a2f/serve-healthcheck-canary/0.log"
Jun 02 05:21:13.577655 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:13.577611    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rbpq7_eb5b0c03-aba6-4531-a125-377ffc9f68e4/kube-rbac-proxy/0.log"
Jun 02 05:21:13.597930 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:13.597909    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rbpq7_eb5b0c03-aba6-4531-a125-377ffc9f68e4/exporter/0.log"
Jun 02 05:21:13.617974 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:13.617956    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rbpq7_eb5b0c03-aba6-4531-a125-377ffc9f68e4/extractor/0.log"
Jun 02 05:21:15.769681 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:15.769631    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-5bb69db85-jblwh_3ac5a2ae-2a3b-4463-a5b5-9f091d343132/nginx/0.log"
Jun 02 05:21:15.792731 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:15.792705    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-7c99ffcdcd-4wxxn_017432b7-cfc9-4dab-8dae-4cd4b48c0479/manager/0.log"
Jun 02 05:21:15.863029 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:15.862993    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-c6b9847d7-fstgz_ae936840-9758-4f32-9197-33c2bf5e7fea/manager/0.log"
Jun 02 05:21:15.918870 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:15.918847    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29672940-p2cx4_aaa45c49-175d-40e0-b626-173babd8f3b1/cleanup/0.log"
Jun 02 05:21:15.939977 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:15.939955    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29672955-x2qhg_5a942609-06c9-4c69-b1f4-b5efe33cd7d7/cleanup/0.log"
Jun 02 05:21:16.035203 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:16.035136    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-6577596cd5-77thv_e007367e-46a0-4676-b34c-b86649eae5ff/server/0.log"
Jun 02 05:21:16.081007 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:16.080991    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-lz5df_3027d5df-6b2f-4eba-8261-5a2f6fb4e98e/manager/0.log"
Jun 02 05:21:16.290324 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:16.290266    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55fcdd494f-rww7z_2e3270e6-5b75-4617-9e28-2e27b883c151/manager/0.log"
Jun 02 05:21:16.313315 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:16.313298    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55fcdd494f-rww7z_2e3270e6-5b75-4617-9e28-2e27b883c151/copy-manifests/0.log"
Jun 02 05:21:16.337209 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:16.337188    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-sng54_ba819424-07f5-4215-98d7-534d6c678735/postgres/0.log"
Jun 02 05:21:17.124355 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:17.124326    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mf6h5/perf-node-gather-daemonset-s2m84"
Jun 02 05:21:17.698895 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:17.698865    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-bs2rj_4473fcda-386d-4aef-a12a-1a7dab6fc1a9/openshift-lws-operator/0.log"
Jun 02 05:21:24.016939 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:24.016872    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29k6t_dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd/kube-multus-additional-cni-plugins/0.log"
Jun 02 05:21:24.040990 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:24.040953    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29k6t_dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd/egress-router-binary-copy/0.log"
Jun 02 05:21:24.060733 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:24.060712    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29k6t_dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd/cni-plugins/0.log"
Jun 02 05:21:24.083794 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:24.083772    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29k6t_dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd/bond-cni-plugin/0.log"
Jun 02 05:21:24.109830 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:24.109794    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29k6t_dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd/routeoverride-cni/0.log"
Jun 02 05:21:24.129258 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:24.129238    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29k6t_dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd/whereabouts-cni-bincopy/0.log"
Jun 02 05:21:24.148785 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:24.148767    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29k6t_dd6cf343-d9aa-452b-9bc8-4f7fc753b6dd/whereabouts-cni/0.log"
Jun 02 05:21:24.744747 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:24.744721    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-t4m5q_8d6a095d-aaa2-4332-b27e-060a5bc84abd/kube-multus/0.log"
Jun 02 05:21:24.767007 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:24.766989    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jnktq_7f26ffcc-136d-49e4-9072-626c6a31246e/network-metrics-daemon/0.log"
Jun 02 05:21:24.788411 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:24.788395    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jnktq_7f26ffcc-136d-49e4-9072-626c6a31246e/kube-rbac-proxy/0.log"
Jun 02 05:21:25.932009 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:25.931980    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-controller/0.log"
Jun 02 05:21:25.948991 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:25.948966    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/0.log"
Jun 02 05:21:25.960045 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:25.960029    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovn-acl-logging/1.log"
Jun 02 05:21:25.978621 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:25.978604    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/kube-rbac-proxy-node/0.log"
Jun 02 05:21:25.997749 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:25.997730    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/kube-rbac-proxy-ovn-metrics/0.log"
Jun 02 05:21:26.015040 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:26.015025    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/northd/0.log"
Jun 02 05:21:26.034351 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:26.034328    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/nbdb/0.log"
Jun 02 05:21:26.053828 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:26.053797    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/sbdb/0.log"
Jun 02 05:21:26.168956 ip-10-0-135-23 kubenswrapper[2577]: I0602 05:21:26.168936    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hgcxc_88d63d7d-55eb-4340-99ae-9c49be3afdbe/ovnkube-controller/0.log"