May 28 17:43:18.752318 ip-10-0-135-41 systemd[1]: Starting Kubernetes Kubelet...
May 28 17:43:19.266368 ip-10-0-135-41 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 17:43:19.266368 ip-10-0-135-41 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 28 17:43:19.266368 ip-10-0-135-41 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 17:43:19.266368 ip-10-0-135-41 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 28 17:43:19.266368 ip-10-0-135-41 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 17:43:19.267362 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.267270    2560 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 28 17:43:19.271481 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271461    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 17:43:19.271481 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271480    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271484    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271488    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271493    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271497    2560 feature_gate.go:328] unrecognized feature gate: Example
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271501    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271505    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271508    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271511    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271513    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271518    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271520    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271523    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271526    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271528    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271531    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271533    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271536    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271538    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 17:43:19.271559 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271541    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271543    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271546    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271548    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271552    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271554    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271557    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271559    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271562    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271565    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271568    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271571    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271574    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271576    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271579    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271581    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271584    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271587    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271589    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271592    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 17:43:19.272035 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271595    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271598    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271600    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271602    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271612    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271615    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271617    2560 feature_gate.go:328] unrecognized feature gate: Example2
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271619    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271622    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271624    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271627    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271629    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271631    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271634    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271638    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271641    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271644    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271646    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271649    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271652    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 17:43:19.272550 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271654    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271657    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271659    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271662    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271666    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271670    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271673    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271675    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271678    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271680    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271683    2560 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271685    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271688    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271690    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271693    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271695    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271698    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271706    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271709    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 17:43:19.273037 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271711    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271714    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271716    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271719    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271721    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271724    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.271726    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272182    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272187    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272190    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272193    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272195    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272198    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272201    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272206    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272209    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272212    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272215    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272218    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 17:43:19.273569 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272221    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272223    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272226    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272229    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272231    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272234    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272236    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272239    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272241    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272244    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272247    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272252    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272255    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272257    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272260    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272262    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272265    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272267    2560 feature_gate.go:328] unrecognized feature gate: Example2
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272270    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 17:43:19.274069 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272272    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272277    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272280    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272282    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272285    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272287    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272290    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272292    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272295    2560 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272297    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272299    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272302    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272304    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272307    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272309    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272312    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272314    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272316    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272319    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272322    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 17:43:19.274646 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272325    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272327    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272331    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272333    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272336    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272339    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272341    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272344    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272347    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272349    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272352    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272354    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272357    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272361    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272363    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272365    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272368    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272371    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272373    2560 feature_gate.go:328] unrecognized feature gate: Example
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272378    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 17:43:19.275164 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272381    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272384    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272386    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272389    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272391    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272394    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272396    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272398    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272401    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272403    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272406    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272409    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272411    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272414    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.272416    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272505    2560 flags.go:64] FLAG: --address="0.0.0.0"
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272516    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272525    2560 flags.go:64] FLAG: --anonymous-auth="true"
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272532    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272537    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272540    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 28 17:43:19.275662 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272545    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272550    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272553    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272557    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272561    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272565    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272569    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272572    2560 flags.go:64] FLAG: --cgroup-root=""
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272575    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272578    2560 flags.go:64] FLAG: --client-ca-file=""
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272581    2560 flags.go:64] FLAG: --cloud-config=""
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272583    2560 flags.go:64] FLAG: --cloud-provider="external"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272586    2560 flags.go:64] FLAG: --cluster-dns="[]"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272591    2560 flags.go:64] FLAG: --cluster-domain=""
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272594    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272598    2560 flags.go:64] FLAG: --config-dir=""
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272600    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272604    2560 flags.go:64] FLAG: --container-log-max-files="5"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272609    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272612    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272615    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272618    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272622    2560 flags.go:64] FLAG: --contention-profiling="false"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272625    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 28 17:43:19.276199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272628    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272631    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272634    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272638    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272641    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272644    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272648    2560 flags.go:64] FLAG: --enable-load-reader="false"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272651    2560 flags.go:64] FLAG: --enable-server="true"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272655    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272659    2560 flags.go:64] FLAG: --event-burst="100"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272662    2560 flags.go:64] FLAG: --event-qps="50"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272665    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272669    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272672    2560 flags.go:64] FLAG: --eviction-hard=""
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272676    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272678    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272681    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272684    2560 flags.go:64] FLAG: --eviction-soft=""
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272687    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272690    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272693    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272696    2560 flags.go:64] FLAG: --experimental-mounter-path=""
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272699    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272702    2560 flags.go:64] FLAG: --fail-swap-on="true"
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272705    2560 flags.go:64] FLAG: --feature-gates=""
May 28 17:43:19.276775 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272712    2560 flags.go:64] FLAG: --file-check-frequency="20s"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272715    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272719    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272722    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272726    2560 flags.go:64] FLAG: --healthz-port="10248"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272729    2560 flags.go:64] FLAG: --help="false"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272732    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-135-41.ec2.internal"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272735    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272739    2560 flags.go:64] FLAG: --http-check-frequency="20s"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272742    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272745    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272749    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272752    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272755    2560 flags.go:64] FLAG: --image-service-endpoint=""
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272758    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272761    2560 flags.go:64] FLAG: --kube-api-burst="100"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272765    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272768    2560 flags.go:64] FLAG: --kube-api-qps="50"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272771    2560 flags.go:64] FLAG: --kube-reserved=""
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272774    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272777    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272780    2560 flags.go:64] FLAG: --kubelet-cgroups=""
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272783    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272786    2560 flags.go:64] FLAG: --lock-file=""
May 28 17:43:19.277407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272789    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272792    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272795    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272800    2560 flags.go:64] FLAG: --log-json-split-stream="false"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272803    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272806    2560 flags.go:64] FLAG: --log-text-split-stream="false"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272809    2560 flags.go:64] FLAG: --logging-format="text"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272812    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272815    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272818    2560 flags.go:64] FLAG: --manifest-url=""
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272821    2560 flags.go:64] FLAG: --manifest-url-header=""
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272825    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272829    2560 flags.go:64] FLAG: --max-open-files="1000000"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272833    2560 flags.go:64] FLAG: --max-pods="110"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272836    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272839    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272842    2560 flags.go:64] FLAG: --memory-manager-policy="None"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272846    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272849    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272852    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272855    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272863    2560 flags.go:64] FLAG: --node-status-max-images="50"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272866    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272869    2560 flags.go:64] FLAG: --oom-score-adj="-999"
May 28 17:43:19.278011 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272872    2560 flags.go:64] FLAG: --pod-cidr=""
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272876    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272882    2560 flags.go:64] FLAG: --pod-manifest-path=""
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272885    2560 flags.go:64] FLAG: --pod-max-pids="-1"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272888    2560 flags.go:64] FLAG: --pods-per-core="0"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272891    2560 flags.go:64] FLAG: --port="10250"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272894    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272897    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-000ca96fab4a50395"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272900    2560 flags.go:64] FLAG: --qos-reserved=""
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272903    2560 flags.go:64] FLAG: --read-only-port="10255"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272906    2560 flags.go:64] FLAG: --register-node="true"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272909    2560 flags.go:64] FLAG: --register-schedulable="true"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272912    2560 flags.go:64] FLAG: --register-with-taints=""
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272916    2560 flags.go:64] FLAG: --registry-burst="10"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272918    2560 flags.go:64] FLAG: --registry-qps="5"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272921    2560 flags.go:64] FLAG: --reserved-cpus=""
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272938    2560 flags.go:64] FLAG: --reserved-memory=""
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272942    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272945    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272948    2560 flags.go:64] FLAG: --rotate-certificates="false"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272952    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272956    2560 flags.go:64] FLAG: --runonce="false"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272959    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272962    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272965    2560 flags.go:64] FLAG: --seccomp-default="false"
May 28 17:43:19.278582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272968    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272971    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272975    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272978    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272981    2560 flags.go:64] FLAG: --storage-driver-password="root"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272984    2560 flags.go:64] FLAG: --storage-driver-secure="false"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272988    2560 flags.go:64] FLAG: --storage-driver-table="stats"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272991    2560 flags.go:64] FLAG: --storage-driver-user="root"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272994    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.272997    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.273000    2560 flags.go:64] FLAG: --system-cgroups=""
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.273003    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.273008    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.273011    2560 flags.go:64] FLAG: --tls-cert-file=""
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.273014    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.273019    2560 flags.go:64] FLAG: --tls-min-version=""
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.273022    2560 flags.go:64] FLAG: --tls-private-key-file=""
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.273025    2560 flags.go:64] FLAG: --topology-manager-policy="none"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.273027    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.273030    2560 flags.go:64] FLAG: --topology-manager-scope="container"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.273033    2560 flags.go:64] FLAG: --v="2"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.273038    2560 flags.go:64] FLAG: --version="false"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.273042    2560 flags.go:64] FLAG: --vmodule=""
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.273047    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.273050    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 28 17:43:19.279191 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273154    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273159    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273163    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273166    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273168    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273171    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273173    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273176    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273179    2560 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273181    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273184    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273187    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273189    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273191    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273195    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273199    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273201    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273204    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273207    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273209    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 17:43:19.279826 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273211    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273214    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273216    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273219    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273221    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273224    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273226    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273229    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273231    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273233    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273236    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273238    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273241    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273247    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273249    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273252    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273255    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273257    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273260    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 17:43:19.280389 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273263    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273266    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273270    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273273    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273276    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273279    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273281    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273284    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273286    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273290    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273292    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273295    2560 feature_gate.go:328] unrecognized feature gate: Example2
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273298    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273300    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273303    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273305    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273308    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273310    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273313    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 17:43:19.280859 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273315    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273318    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273320    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273323    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273326    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273328    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273330    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273335    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273337    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273340    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273342    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273345    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273347    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273352    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273355    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273357    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273360    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273362    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273365    2560 feature_gate.go:328] unrecognized feature gate: Example
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273368    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 17:43:19.281345 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273370    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 17:43:19.281846 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273373    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 17:43:19.281846 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273376    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 17:43:19.281846 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273380    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 17:43:19.281846 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273383    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 17:43:19.281846 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273386    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 17:43:19.281846 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273389    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 17:43:19.281846 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.273392    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 17:43:19.281846 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.274207    2560 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 17:43:19.283564 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.283537    2560 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 28 17:43:19.283564 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.283560    2560 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283614    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283620    2560 feature_gate.go:328] unrecognized feature gate: Example
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283624    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283627    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283631    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283634    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283637    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283640    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283642    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283645    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283648    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283651    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283653    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283656    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283659    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283661    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283664    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283666    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283669    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 17:43:19.283701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283671    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283674    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283677    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283679    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283681    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283684    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283686    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283689    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283691    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283695    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283698    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283701    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283703    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283706    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283709    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283712    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283714    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283718    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283722    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 17:43:19.284281 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283725    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283727    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283730    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283732    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283735    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283737    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283740    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283744    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283748    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283751    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283753    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283756    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283759    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283761    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283763    2560 feature_gate.go:328] unrecognized feature gate: Example2
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283767    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283770    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283773    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283776    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283778    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 17:43:19.284790 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283781    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283783    2560 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283786    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283789    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283791    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283793    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283796    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283798    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283801    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283804    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283806    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283808    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283811    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283814    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283816    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283819    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283821    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283823    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283827    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283829    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 17:43:19.285352 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283831    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 17:43:19.285900 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283834    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 17:43:19.285900 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283837    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 17:43:19.285900 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283839    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 17:43:19.285900 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283842    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 17:43:19.285900 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283844    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 17:43:19.285900 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283847    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 17:43:19.285900 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283850    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 17:43:19.285900 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.283855    2560 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 17:43:19.285900 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283964    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 17:43:19.285900 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283970    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 17:43:19.285900 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283973    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 17:43:19.285900 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283975    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 17:43:19.285900 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283978    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 17:43:19.285900 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283981    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 17:43:19.285900 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283983    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 17:43:19.285900 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283986    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283988    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283991    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283994    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283996    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.283998    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284001    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284004    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284006    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284009    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284011    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284014    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284016    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284019    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284021    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284025    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284027    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284030    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284032    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284035    2560 feature_gate.go:328] unrecognized feature gate: Example
May 28 17:43:19.286363 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284037    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284040    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284042    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284045    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284048    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284051    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284053    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284056    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284059    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284061    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284064    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284067    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284071    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284074    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284076    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284079    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284082    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284084    2560 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 17:43:19.286893 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284087    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284090    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284094    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284097    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284100    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284102    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284105    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284107    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284110    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284112    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284116    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284118    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284121    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284123    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284125    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284128    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284131    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284133    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284136    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 17:43:19.287357 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284139    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284142    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284144    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284147    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284149    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284152    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284154    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284157    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284159    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284162    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284164    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284167    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284169    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284172    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284174    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284177    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284180    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284182    2560 feature_gate.go:328] unrecognized feature gate: Example2
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284185    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284187    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 17:43:19.287830 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284190    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 17:43:19.288451 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:19.284192    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 17:43:19.288451 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.284197    2560 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 17:43:19.288451 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.285034    2560 server.go:962] "Client rotation is on, will bootstrap in background"
May 28 17:43:19.289135 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.289120    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 28 17:43:19.290268 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.290255    2560 server.go:1019] "Starting client certificate rotation"
May 28 17:43:19.290374 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.290355    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 17:43:19.290408 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.290393    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 17:43:19.320319 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.320294    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 17:43:19.322290 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.322271    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 17:43:19.340981 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.340956    2560 log.go:25] "Validated CRI v1 runtime API"
May 28 17:43:19.347296 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.347276    2560 log.go:25] "Validated CRI v1 image API"
May 28 17:43:19.348595 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.348566    2560 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 17:43:19.348836 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.348823    2560 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 28 17:43:19.351980 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.351960    2560 fs.go:135] Filesystem UUIDs: map[061f0e0b-f7e8-4b74-bc71-f2b7625df710:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 7a77f75c-b7d5-4315-affa-15580838ea23:/dev/nvme0n1p3]
May 28 17:43:19.352054 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.351980    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 28 17:43:19.357429 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.357305    2560 manager.go:217] Machine: {Timestamp:2026-05-28 17:43:19.35675946 +0000 UTC m=+0.460455169 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3112789 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20bee0200b55be85dddf2bc6a0519c SystemUUID:ec20bee0-200b-55be-85dd-df2bc6a0519c BootID:32eb79d2-6c64-435f-a54d-8d350e24c95d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:60:45:bd:fe:d9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:60:45:bd:fe:d9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:14:89:71:68:46 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 28 17:43:19.357429 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.357418    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 28 17:43:19.357572 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.357503    2560 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 28 17:43:19.361649 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.361615    2560 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 28 17:43:19.361796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.361654    2560 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-41.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 28 17:43:19.361866 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.361807    2560 topology_manager.go:138] "Creating topology manager with none policy"
May 28 17:43:19.361866 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.361816    2560 container_manager_linux.go:306] "Creating device plugin manager"
May 28 17:43:19.361866 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.361824    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 17:43:19.361866 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.361840    2560 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 17:43:19.364484 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.364472    2560 state_mem.go:36] "Initialized new in-memory state store"
May 28 17:43:19.364792 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.364782    2560 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 28 17:43:19.367586 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.367575    2560 kubelet.go:491] "Attempting to sync node with API server"
May 28 17:43:19.367621 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.367594    2560 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 28 17:43:19.368394 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.368384    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 28 17:43:19.368446 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.368397    2560 kubelet.go:397] "Adding apiserver pod source"
May 28 17:43:19.368446 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.368407    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 28 17:43:19.369734 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.369715    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 17:43:19.369734 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.369735    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 17:43:19.375598 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.375579    2560 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 28 17:43:19.378837 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.378818    2560 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 28 17:43:19.379249 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.379227    2560 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 28 17:43:19.379334 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.379296    2560 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-41.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 28 17:43:19.379805 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.379794    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 28 17:43:19.379842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.379811    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 28 17:43:19.379842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.379817    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 28 17:43:19.379842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.379824    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 28 17:43:19.379842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.379832    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 28 17:43:19.379842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.379837    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 28 17:43:19.379842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.379843    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 28 17:43:19.380023 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.379849    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 28 17:43:19.380023 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.379857    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 28 17:43:19.380023 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.379863    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 28 17:43:19.380023 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.379872    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 28 17:43:19.380300 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.380290    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 28 17:43:19.381280 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.381270    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 28 17:43:19.381316 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.381281    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 28 17:43:19.384858 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.384843    2560 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 28 17:43:19.384936 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.384884    2560 server.go:1295] "Started kubelet"
May 28 17:43:19.385025 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.384992    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 28 17:43:19.385135 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.385014    2560 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 28 17:43:19.385206 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.385190    2560 server_v1.go:47] "podresources" method="list" useActivePods=true
May 28 17:43:19.385695 ip-10-0-135-41 systemd[1]: Started Kubernetes Kubelet.
May 28 17:43:19.386740 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.386720    2560 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 28 17:43:19.389452 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.389438    2560 server.go:317] "Adding debug handlers to kubelet server"
May 28 17:43:19.392816 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.392799    2560 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-41.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 28 17:43:19.394032 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.392831    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-41.ec2.internal.18b3cba7d202ddf2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-41.ec2.internal,UID:ip-10-0-135-41.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-41.ec2.internal,},FirstTimestamp:2026-05-28 17:43:19.384858098 +0000 UTC m=+0.488553806,LastTimestamp:2026-05-28 17:43:19.384858098 +0000 UTC m=+0.488553806,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-41.ec2.internal,}"
May 28 17:43:19.395832 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.395810    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 28 17:43:19.396417 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.396400    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 28 17:43:19.396670 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.396648    2560 volume_manager.go:295] "The desired_state_of_world populator starts"
May 28 17:43:19.396670 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.396674    2560 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 28 17:43:19.396891 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.396765    2560 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 28 17:43:19.396891 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.396830    2560 reconstruct.go:97] "Volume reconstruction finished"
May 28 17:43:19.396891 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.396838    2560 reconciler.go:26] "Reconciler: start to sync state"
May 28 17:43:19.398015 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.397992    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-41.ec2.internal\" not found"
May 28 17:43:19.398176 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.398134    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jtxjc"
May 28 17:43:19.398295 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.398278    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 28 17:43:19.398357 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.398299    2560 factory.go:55] Registering systemd factory
May 28 17:43:19.398357 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.398309    2560 factory.go:223] Registration of the systemd container factory successfully
May 28 17:43:19.398711 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.398687    2560 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
May 28 17:43:19.398826 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.398798    2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-41.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
May 28 17:43:19.398890 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.398879    2560 factory.go:153] Registering CRI-O factory
May 28 17:43:19.398959 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.398895    2560 factory.go:223] Registration of the crio container factory successfully
May 28 17:43:19.398959 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.398913    2560 factory.go:103] Registering Raw factory
May 28 17:43:19.398959 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.398949    2560 manager.go:1196] Started watching for new ooms in manager
May 28 17:43:19.400016 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.399481    2560 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 28 17:43:19.400124 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.400111    2560 manager.go:319] Starting recovery of all containers
May 28 17:43:19.403582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.403558    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jtxjc"
May 28 17:43:19.411254 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.411124    2560 manager.go:324] Recovery completed
May 28 17:43:19.415560 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.415545    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:43:19.417530 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.417512    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:43:19.417604 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.417547    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:43:19.417604 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.417562    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasSufficientPID"
May 28 17:43:19.418198 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.418185    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 28 17:43:19.418266 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.418197    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 28 17:43:19.418266 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.418213    2560 state_mem.go:36] "Initialized new in-memory state store"
May 28 17:43:19.421984 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.421972    2560 policy_none.go:49] "None policy: Start"
May 28 17:43:19.422026 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.421996    2560 memory_manager.go:186] "Starting memorymanager" policy="None"
May 28 17:43:19.422026 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.422007    2560 state_mem.go:35] "Initializing new in-memory state store"
May 28 17:43:19.470602 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.470583    2560 manager.go:341] "Starting Device Plugin manager"
May 28 17:43:19.495772 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.470626    2560 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 28 17:43:19.495772 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.470637    2560 server.go:85] "Starting device plugin registration server"
May 28 17:43:19.495772 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.470921    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
May 28 17:43:19.495772 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.470955    2560 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 28 17:43:19.495772 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.471060    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 28 17:43:19.495772 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.471144    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 28 17:43:19.495772 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.471152    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 28 17:43:19.495772 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.471662    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 28 17:43:19.495772 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.471697    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-41.ec2.internal\" not found"
May 28 17:43:19.511623 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.511583    2560 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 28 17:43:19.512976 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.512959    2560 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 28 17:43:19.513041 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.512985    2560 status_manager.go:230] "Starting to sync pod status with apiserver"
May 28 17:43:19.513041 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.513005    2560 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 28 17:43:19.513041 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.513012    2560 kubelet.go:2451] "Starting kubelet main sync loop"
May 28 17:43:19.513166 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.513050    2560 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 28 17:43:19.515216 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.515191    2560 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 17:43:19.571275 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.571244    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:43:19.572239 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.572220    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:43:19.572360 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.572257    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:43:19.572360 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.572272    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasSufficientPID"
May 28 17:43:19.572360 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.572303    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-41.ec2.internal"
May 28 17:43:19.584659 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.584637    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-41.ec2.internal"
May 28 17:43:19.584741 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.584668    2560 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-41.ec2.internal\": node \"ip-10-0-135-41.ec2.internal\" not found"
May 28 17:43:19.596261 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.596235    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-41.ec2.internal\" not found"
May 28 17:43:19.613619 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.613591    2560 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-41.ec2.internal"]
May 28 17:43:19.613689 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.613662    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:43:19.614520 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.614502    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:43:19.614609 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.614535    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:43:19.614609 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.614546    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasSufficientPID"
May 28 17:43:19.615694 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.615682    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:43:19.615903 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.615891    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal"
May 28 17:43:19.615956 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.615922    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:43:19.618431 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.618416    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:43:19.618519 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.618440    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:43:19.618519 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.618430    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:43:19.618519 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.618450    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasSufficientPID"
May 28 17:43:19.618519 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.618470    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:43:19.618519 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.618485    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasSufficientPID"
May 28 17:43:19.619391 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.619378    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-41.ec2.internal"
May 28 17:43:19.619436 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.619411    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:43:19.620136 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.620121    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:43:19.620199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.620150    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:43:19.620199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.620160    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeHasSufficientPID"
May 28 17:43:19.634477 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.634458    2560 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-41.ec2.internal\" not found" node="ip-10-0-135-41.ec2.internal"
May 28 17:43:19.638433 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.638418    2560 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-41.ec2.internal\" not found" node="ip-10-0-135-41.ec2.internal"
May 28 17:43:19.696844 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.696809    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-41.ec2.internal\" not found"
May 28 17:43:19.698010 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.697992    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7836682e045d93471bf2a629158ba14-config\") pod \"kube-apiserver-proxy-ip-10-0-135-41.ec2.internal\" (UID: \"a7836682e045d93471bf2a629158ba14\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-41.ec2.internal"
May 28 17:43:19.698064 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.698025    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/43f38dd408635a9aab60a231c225b6f8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal\" (UID: \"43f38dd408635a9aab60a231c225b6f8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal"
May 28 17:43:19.698064 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.698049    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43f38dd408635a9aab60a231c225b6f8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal\" (UID: \"43f38dd408635a9aab60a231c225b6f8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal"
May 28 17:43:19.797147 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.797116    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-41.ec2.internal\" not found"
May 28 17:43:19.798270 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.798248    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/43f38dd408635a9aab60a231c225b6f8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal\" (UID: \"43f38dd408635a9aab60a231c225b6f8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal"
May 28 17:43:19.798324 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.798281    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43f38dd408635a9aab60a231c225b6f8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal\" (UID: \"43f38dd408635a9aab60a231c225b6f8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal"
May 28 17:43:19.798324 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.798298    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7836682e045d93471bf2a629158ba14-config\") pod \"kube-apiserver-proxy-ip-10-0-135-41.ec2.internal\" (UID: \"a7836682e045d93471bf2a629158ba14\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-41.ec2.internal"
May 28 17:43:19.798391 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.798334    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43f38dd408635a9aab60a231c225b6f8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal\" (UID: \"43f38dd408635a9aab60a231c225b6f8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal"
May 28 17:43:19.798391 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.798338    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7836682e045d93471bf2a629158ba14-config\") pod \"kube-apiserver-proxy-ip-10-0-135-41.ec2.internal\" (UID: \"a7836682e045d93471bf2a629158ba14\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-41.ec2.internal"
May 28 17:43:19.798464 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.798396    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/43f38dd408635a9aab60a231c225b6f8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal\" (UID: \"43f38dd408635a9aab60a231c225b6f8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal"
May 28 17:43:19.897710 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.897634    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-41.ec2.internal\" not found"
May 28 17:43:19.936884 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.936851    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal"
May 28 17:43:19.940654 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:19.940636    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-41.ec2.internal"
May 28 17:43:19.998444 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:19.998408    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-41.ec2.internal\" not found"
May 28 17:43:20.098833 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:20.098815    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-41.ec2.internal\" not found"
May 28 17:43:20.199366 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:20.199301    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-41.ec2.internal\" not found"
May 28 17:43:20.289536 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.289505    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 28 17:43:20.290137 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.289646    2560 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 17:43:20.299814 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:20.299792    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-41.ec2.internal\" not found"
May 28 17:43:20.319108 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.319089    2560 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 17:43:20.369008 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.368987    2560 apiserver.go:52] "Watching apiserver"
May 28 17:43:20.380155 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.380132    2560 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 28 17:43:20.380433 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.380412    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-xfjx8","openshift-ovn-kubernetes/ovnkube-node-fsvt7","kube-system/konnectivity-agent-96mcv","openshift-cluster-node-tuning-operator/tuned-mk47q","openshift-dns/node-resolver-5mv2t","openshift-multus/multus-27ntt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk","openshift-image-registry/node-ca-46l5q","openshift-multus/multus-additional-cni-plugins-k47r7","openshift-multus/network-metrics-daemon-rh246","openshift-network-diagnostics/network-check-target-ch7tt"]
May 28 17:43:20.382646 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.382625    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xfjx8"
May 28 17:43:20.384882 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.384862    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 28 17:43:20.385035 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.385017    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rdxpg\""
May 28 17:43:20.385453 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.385157    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 28 17:43:20.385707 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.385687    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.386012 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.385825    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 28 17:43:20.387922 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.387900    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 28 17:43:20.388126 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.388108    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-96mcv"
May 28 17:43:20.388196 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.388166    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 28 17:43:20.388248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.388238    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 28 17:43:20.388299 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.388267    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 28 17:43:20.388378 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.388349    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pcj2d\""
May 28 17:43:20.388378 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.388363    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 28 17:43:20.388469 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.388451    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 28 17:43:20.390034 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.390017    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.390128 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.390116    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 28 17:43:20.390219 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.390203    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dwtzx\""
May 28 17:43:20.390437 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.390414    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 28 17:43:20.392060 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.392042    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-b64f4\""
May 28 17:43:20.392139 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.392131    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 28 17:43:20.392225 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.392208    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 28 17:43:20.392305 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.392292    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5mv2t"
May 28 17:43:20.394416 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.394399    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 28 17:43:20.394501 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.394407    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wf9zl\""
May 28 17:43:20.394501 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.394439    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 28 17:43:20.394739 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.394717    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-27ntt"
May 28 17:43:20.396138 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.396121    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 28 17:43:20.396572 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.396556    2560 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal"
May 28 17:43:20.396713 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.396699    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 28 17:43:20.396973 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.396956    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 28 17:43:20.397057 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.396975    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 28 17:43:20.397126 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.397064    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 28 17:43:20.397126 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.397072    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8lrq8\""
May 28 17:43:20.397361 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.397346    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.399877 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.399835    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 28 17:43:20.400017 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.399846    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 28 17:43:20.400124 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.400034    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vm4kr\""
May 28 17:43:20.400238 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.400223    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 28 17:43:20.401008 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.400989    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-sys\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.401091 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401015    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-hostroot\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.401091 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401034    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/538aa87a-ba23-4a27-80d9-73321fcf04a5-iptables-alerter-script\") pod \"iptables-alerter-xfjx8\" (UID: \"538aa87a-ba23-4a27-80d9-73321fcf04a5\") " pod="openshift-network-operator/iptables-alerter-xfjx8"
May 28 17:43:20.401091 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401060    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-run-openvswitch\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.401241 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401092    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-run-ovn\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.401241 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401153    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-run-netns\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.401241 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401188    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-systemd-units\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.401241 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401215    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-run-netns\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.401415 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401241    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.401415 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401277    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90d40580-f219-42b1-9b78-9c97e23cf541-etc-tuned\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.401415 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401311    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-run-systemd\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.401415 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401339    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-run-ovn-kubernetes\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.402201 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401695    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70ec7ff0-1089-4ca1-8238-da94d4b1c031-tmp-dir\") pod \"node-resolver-5mv2t\" (UID: \"70ec7ff0-1089-4ca1-8238-da94d4b1c031\") " pod="openshift-dns/node-resolver-5mv2t"
May 28 17:43:20.402201 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401767    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5m5m\" (UniqueName: \"kubernetes.io/projected/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-kube-api-access-q5m5m\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.402201 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401795    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f94fa363-005b-460a-adea-49116e3b4776-agent-certs\") pod \"konnectivity-agent-96mcv\" (UID: \"f94fa363-005b-460a-adea-49116e3b4776\") " pod="kube-system/konnectivity-agent-96mcv"
May 28 17:43:20.402201 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401831    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-sysctl-conf\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.402201 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401860    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-var-lib-kubelet\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.402201 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401889    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-multus-conf-dir\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.402201 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.401919    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29bzf\" (UniqueName: \"kubernetes.io/projected/70ec7ff0-1089-4ca1-8238-da94d4b1c031-kube-api-access-29bzf\") pod \"node-resolver-5mv2t\" (UID: \"70ec7ff0-1089-4ca1-8238-da94d4b1c031\") " pod="openshift-dns/node-resolver-5mv2t"
May 28 17:43:20.402201 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.402004    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-cnibin\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.402201 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.402061    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-multus-daemon-config\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.402201 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.402094    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-var-lib-openvswitch\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.402201 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.402126    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-etc-openvswitch\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.402201 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.402193    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-log-socket\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.402694 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.402227    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.402694 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.402253    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qctrm\" (UniqueName: \"kubernetes.io/projected/b2eba6df-3e3e-46c8-b3c8-a1160503d901-kube-api-access-qctrm\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.402694 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.402278    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f94fa363-005b-460a-adea-49116e3b4776-konnectivity-ca\") pod \"konnectivity-agent-96mcv\" (UID: \"f94fa363-005b-460a-adea-49116e3b4776\") " pod="kube-system/konnectivity-agent-96mcv"
May 28 17:43:20.402694 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.402304    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-sysconfig\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.402694 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.402341    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-multus-socket-dir-parent\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.402866 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.402846    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt8sh\" (UniqueName: \"kubernetes.io/projected/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-kube-api-access-qt8sh\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.402923 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.402891    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-registration-dir\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.402997 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.402922    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-device-dir\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.402997 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.402971    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-host\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.403098 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.402996    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-var-lib-cni-bin\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.403145 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.403117    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b2eba6df-3e3e-46c8-b3c8-a1160503d901-ovnkube-script-lib\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.403190 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.403149    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-systemd\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.403190 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.403181    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b2eba6df-3e3e-46c8-b3c8-a1160503d901-env-overrides\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.403291 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.403225    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.403291 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.403254    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-run-k8s-cni-cncf-io\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.403389 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.403282    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-cni-bin\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.403389 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.403374    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-run-multus-certs\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.403486 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.403409    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/538aa87a-ba23-4a27-80d9-73321fcf04a5-host-slash\") pod \"iptables-alerter-xfjx8\" (UID: \"538aa87a-ba23-4a27-80d9-73321fcf04a5\") " pod="openshift-network-operator/iptables-alerter-xfjx8"
May 28 17:43:20.403609 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.403590    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70ec7ff0-1089-4ca1-8238-da94d4b1c031-hosts-file\") pod \"node-resolver-5mv2t\" (UID: \"70ec7ff0-1089-4ca1-8238-da94d4b1c031\") " pod="openshift-dns/node-resolver-5mv2t"
May 28 17:43:20.403659 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.403627    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-socket-dir\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.404235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.403857    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-sys-fs\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.404235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.403897    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-run\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.404235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404079    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-lib-modules\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.404235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404109    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-os-release\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.404235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404121    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-46l5q"
May 28 17:43:20.404235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404139    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-var-lib-cni-multus\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.404235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404171    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-etc-kubernetes\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.404235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404203    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww24b\" (UniqueName: \"kubernetes.io/projected/538aa87a-ba23-4a27-80d9-73321fcf04a5-kube-api-access-ww24b\") pod \"iptables-alerter-xfjx8\" (UID: \"538aa87a-ba23-4a27-80d9-73321fcf04a5\") " pod="openshift-network-operator/iptables-alerter-xfjx8"
May 28 17:43:20.404235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404228    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-slash\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.404524 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404254    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90d40580-f219-42b1-9b78-9c97e23cf541-tmp\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.404524 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404284    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b2eba6df-3e3e-46c8-b3c8-a1160503d901-ovn-node-metrics-cert\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.404524 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404321    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-multus-cni-dir\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.404524 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404350    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-kubelet\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.404524 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404379    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-kubernetes\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.404524 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404411    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b2eba6df-3e3e-46c8-b3c8-a1160503d901-ovnkube-config\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.404524 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404441    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-modprobe-d\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.404748 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404463    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz9ft\" (UniqueName: \"kubernetes.io/projected/90d40580-f219-42b1-9b78-9c97e23cf541-kube-api-access-zz9ft\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.404748 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404580    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-system-cni-dir\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.404748 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404610    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-sysctl-d\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.404748 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404641    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-cni-binary-copy\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.404748 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404669    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-var-lib-kubelet\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.404748 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404697    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-node-log\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.404748 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.404723    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-cni-netd\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.405550 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.405532    2560 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 17:43:20.406403 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.405848    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 17:38:19 +0000 UTC" deadline="2028-01-20 18:43:25.258527426 +0000 UTC"
May 28 17:43:20.406403 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.405900    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14449h0m4.852631288s"
May 28 17:43:20.406911 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.406896    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 28 17:43:20.407106 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.406897    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 28 17:43:20.407106 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.406967    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sx9fv\""
May 28 17:43:20.407216 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.407154    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 28 17:43:20.408514 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.408482    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.408615 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.408599    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:20.408680 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:20.408659    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:20.410511 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.410496    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 28 17:43:20.410600 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.410522    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 28 17:43:20.410690 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.410678    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gkwhx\""
May 28 17:43:20.411019 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.411003    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:20.411071 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:20.411055    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:20.412337 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.412318    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal"]
May 28 17:43:20.413065 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.413051    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 17:43:20.413129 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.413115    2560 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-41.ec2.internal"
May 28 17:43:20.419267 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.419250    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 17:43:20.419400 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.419345    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-41.ec2.internal"]
May 28 17:43:20.430320 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.430303    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-t2bk2"
May 28 17:43:20.439196 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.439073    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-t2bk2"
May 28 17:43:20.497793 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.497768    2560 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 28 17:43:20.505438 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.505415    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90d40580-f219-42b1-9b78-9c97e23cf541-tmp\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.505531 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.505471    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b2eba6df-3e3e-46c8-b3c8-a1160503d901-ovn-node-metrics-cert\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.505531 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.505513    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f454ed50-48e4-4202-88d3-1402e75de767-cni-binary-copy\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.505609 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.505542    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-multus-cni-dir\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.505609 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.505567    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-kubelet\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.505609 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.505591    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-kubernetes\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.505716 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.505614    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b2eba6df-3e3e-46c8-b3c8-a1160503d901-ovnkube-config\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.505716 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.505641    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-multus-cni-dir\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.505716 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.505671    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-modprobe-d\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.505716 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.505687    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-kubernetes\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.505716 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.505696    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zz9ft\" (UniqueName: \"kubernetes.io/projected/90d40580-f219-42b1-9b78-9c97e23cf541-kube-api-access-zz9ft\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.505898 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.505725    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-system-cni-dir\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.505898 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.505753    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ec78263b-aee2-442b-8bc0-5f738c3d6832-serviceca\") pod \"node-ca-46l5q\" (UID: \"ec78263b-aee2-442b-8bc0-5f738c3d6832\") " pod="openshift-image-registry/node-ca-46l5q"
May 28 17:43:20.506358 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506027    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-modprobe-d\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.506358 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506088    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 28 17:43:20.506358 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506109    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-sysctl-d\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.506358 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.505768    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-kubelet\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.506594 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506405    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-sysctl-d\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.506594 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506424    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-system-cni-dir\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.506594 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506454    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-cni-binary-copy\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.506594 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506472    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b2eba6df-3e3e-46c8-b3c8-a1160503d901-ovnkube-config\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.506594 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506561    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-var-lib-kubelet\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.506842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506593    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-node-log\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.506842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506644    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-node-log\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.506842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506663    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-cni-netd\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.506842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506669    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-var-lib-kubelet\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.506842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506710    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec78263b-aee2-442b-8bc0-5f738c3d6832-host\") pod \"node-ca-46l5q\" (UID: \"ec78263b-aee2-442b-8bc0-5f738c3d6832\") " pod="openshift-image-registry/node-ca-46l5q"
May 28 17:43:20.506842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506710    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-cni-netd\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.506842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506740    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-sys\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.506842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506770    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-hostroot\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.506842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506798    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/538aa87a-ba23-4a27-80d9-73321fcf04a5-iptables-alerter-script\") pod \"iptables-alerter-xfjx8\" (UID: \"538aa87a-ba23-4a27-80d9-73321fcf04a5\") " pod="openshift-network-operator/iptables-alerter-xfjx8"
May 28 17:43:20.506842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506810    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-sys\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.506842 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506838    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-run-openvswitch\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506863    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-hostroot\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506870    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-run-ovn\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506907    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqvm6\" (UniqueName: \"kubernetes.io/projected/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-kube-api-access-bqvm6\") pod \"network-metrics-daemon-rh246\" (UID: \"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4\") " pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506961    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-run-netns\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.506991    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-systemd-units\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507022    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-run-netns\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507053    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507089    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x62nf\" (UniqueName: \"kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf\") pod \"network-check-target-ch7tt\" (UID: \"0eec8ff8-0b7b-4d4d-a787-463843df5ef7\") " pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507124    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90d40580-f219-42b1-9b78-9c97e23cf541-etc-tuned\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507150    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-run-systemd\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507184    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-run-ovn-kubernetes\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507218    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70ec7ff0-1089-4ca1-8238-da94d4b1c031-tmp-dir\") pod \"node-resolver-5mv2t\" (UID: \"70ec7ff0-1089-4ca1-8238-da94d4b1c031\") " pod="openshift-dns/node-resolver-5mv2t"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507253    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q5m5m\" (UniqueName: \"kubernetes.io/projected/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-kube-api-access-q5m5m\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507289    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f454ed50-48e4-4202-88d3-1402e75de767-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507318    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f94fa363-005b-460a-adea-49116e3b4776-agent-certs\") pod \"konnectivity-agent-96mcv\" (UID: \"f94fa363-005b-460a-adea-49116e3b4776\") " pod="kube-system/konnectivity-agent-96mcv"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507347    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-sysctl-conf\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.507406 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507376    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-var-lib-kubelet\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.508235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507409    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-multus-conf-dir\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.508235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507418    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/538aa87a-ba23-4a27-80d9-73321fcf04a5-iptables-alerter-script\") pod \"iptables-alerter-xfjx8\" (UID: \"538aa87a-ba23-4a27-80d9-73321fcf04a5\") " pod="openshift-network-operator/iptables-alerter-xfjx8"
May 28 17:43:20.508235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507429    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-cni-binary-copy\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.508235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507445    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-29bzf\" (UniqueName: \"kubernetes.io/projected/70ec7ff0-1089-4ca1-8238-da94d4b1c031-kube-api-access-29bzf\") pod \"node-resolver-5mv2t\" (UID: \"70ec7ff0-1089-4ca1-8238-da94d4b1c031\") " pod="openshift-dns/node-resolver-5mv2t"
May 28 17:43:20.508235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507505    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-run-openvswitch\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.508235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507762    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70ec7ff0-1089-4ca1-8238-da94d4b1c031-tmp-dir\") pod \"node-resolver-5mv2t\" (UID: \"70ec7ff0-1089-4ca1-8238-da94d4b1c031\") " pod="openshift-dns/node-resolver-5mv2t"
May 28 17:43:20.508235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507794    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-multus-conf-dir\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.508235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.507917    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-sysctl-conf\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.508235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508007    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-var-lib-kubelet\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.508235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508073    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-systemd-units\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.508235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508129    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-run-ovn\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.508235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508190    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-run-netns\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.508235 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508235    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-run-systemd\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.509101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508469    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-run-netns\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.509101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508564    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-etc-selinux\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.509101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508621    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-run-ovn-kubernetes\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.509101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508657    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f454ed50-48e4-4202-88d3-1402e75de767-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.509101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508694    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f454ed50-48e4-4202-88d3-1402e75de767-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.509101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508730    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-cnibin\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.509101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508764    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-multus-daemon-config\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.509101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508797    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-var-lib-openvswitch\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.509101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508826    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-etc-openvswitch\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.509101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508858    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-log-socket\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.509101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508890    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.509101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.508988    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qctrm\" (UniqueName: \"kubernetes.io/projected/b2eba6df-3e3e-46c8-b3c8-a1160503d901-kube-api-access-qctrm\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.509101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509028    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6956\" (UniqueName: \"kubernetes.io/projected/f454ed50-48e4-4202-88d3-1402e75de767-kube-api-access-l6956\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.509101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509058    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f94fa363-005b-460a-adea-49116e3b4776-konnectivity-ca\") pod \"konnectivity-agent-96mcv\" (UID: \"f94fa363-005b-460a-adea-49116e3b4776\") " pod="kube-system/konnectivity-agent-96mcv"
May 28 17:43:20.509101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509087    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-sysconfig\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509118    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-multus-socket-dir-parent\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509149    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qt8sh\" (UniqueName: \"kubernetes.io/projected/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-kube-api-access-qt8sh\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509181    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-registration-dir\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509207    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-device-dir\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509240    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knh6s\" (UniqueName: \"kubernetes.io/projected/ec78263b-aee2-442b-8bc0-5f738c3d6832-kube-api-access-knh6s\") pod \"node-ca-46l5q\" (UID: \"ec78263b-aee2-442b-8bc0-5f738c3d6832\") " pod="openshift-image-registry/node-ca-46l5q"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509271    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f454ed50-48e4-4202-88d3-1402e75de767-system-cni-dir\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509315    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-host\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509344    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-var-lib-cni-bin\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509370    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b2eba6df-3e3e-46c8-b3c8-a1160503d901-ovnkube-script-lib\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509405    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f454ed50-48e4-4202-88d3-1402e75de767-cnibin\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509436    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f454ed50-48e4-4202-88d3-1402e75de767-os-release\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509471    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-systemd\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509515    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b2eba6df-3e3e-46c8-b3c8-a1160503d901-env-overrides\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509553    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509594    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs\") pod \"network-metrics-daemon-rh246\" (UID: \"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4\") " pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509640    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-run-k8s-cni-cncf-io\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.509796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509669    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-cni-bin\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509703    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-run-multus-certs\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509737    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/538aa87a-ba23-4a27-80d9-73321fcf04a5-host-slash\") pod \"iptables-alerter-xfjx8\" (UID: \"538aa87a-ba23-4a27-80d9-73321fcf04a5\") " pod="openshift-network-operator/iptables-alerter-xfjx8"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509772    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70ec7ff0-1089-4ca1-8238-da94d4b1c031-hosts-file\") pod \"node-resolver-5mv2t\" (UID: \"70ec7ff0-1089-4ca1-8238-da94d4b1c031\") " pod="openshift-dns/node-resolver-5mv2t"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509805    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-socket-dir\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509834    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90d40580-f219-42b1-9b78-9c97e23cf541-tmp\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509911    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-sys-fs\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509920    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-cnibin\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509994    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b2eba6df-3e3e-46c8-b3c8-a1160503d901-ovn-node-metrics-cert\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.509836    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-sys-fs\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.510047    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-run\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.510083    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-lib-modules\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.510117    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-os-release\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.510144    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-var-lib-cni-multus\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.510199    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-etc-kubernetes\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.510236    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ww24b\" (UniqueName: \"kubernetes.io/projected/538aa87a-ba23-4a27-80d9-73321fcf04a5-kube-api-access-ww24b\") pod \"iptables-alerter-xfjx8\" (UID: \"538aa87a-ba23-4a27-80d9-73321fcf04a5\") " pod="openshift-network-operator/iptables-alerter-xfjx8"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.510271    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-slash\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.510294    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90d40580-f219-42b1-9b78-9c97e23cf541-etc-tuned\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.510755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.510447    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-slash\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.510557    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-sysconfig\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.510617    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-multus-socket-dir-parent\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.510905    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-registration-dir\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.510912    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-multus-daemon-config\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.510984    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-var-lib-openvswitch\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.510997    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-host\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511016    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f94fa363-005b-460a-adea-49116e3b4776-agent-certs\") pod \"konnectivity-agent-96mcv\" (UID: \"f94fa363-005b-460a-adea-49116e3b4776\") " pod="kube-system/konnectivity-agent-96mcv"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511034    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-device-dir\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511046    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-var-lib-cni-bin\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511097    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-run-k8s-cni-cncf-io\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511120    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511126    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-host-cni-bin\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511115    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/538aa87a-ba23-4a27-80d9-73321fcf04a5-host-slash\") pod \"iptables-alerter-xfjx8\" (UID: \"538aa87a-ba23-4a27-80d9-73321fcf04a5\") " pod="openshift-network-operator/iptables-alerter-xfjx8"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511176    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-run-multus-certs\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511183    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-run\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511203    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70ec7ff0-1089-4ca1-8238-da94d4b1c031-hosts-file\") pod \"node-resolver-5mv2t\" (UID: \"70ec7ff0-1089-4ca1-8238-da94d4b1c031\") " pod="openshift-dns/node-resolver-5mv2t"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511242    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-os-release\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.512248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511282    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-host-var-lib-cni-multus\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.513080 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511293    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-socket-dir\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.513080 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511321    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-etc-kubernetes\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.513080 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511369    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-etc-systemd\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.513080 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511427    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90d40580-f219-42b1-9b78-9c97e23cf541-lib-modules\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.513080 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511453    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-log-socket\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.513080 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511522    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b2eba6df-3e3e-46c8-b3c8-a1160503d901-env-overrides\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.513080 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511527    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.513080 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511579    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b2eba6df-3e3e-46c8-b3c8-a1160503d901-etc-openvswitch\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.513080 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511802    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f94fa363-005b-460a-adea-49116e3b4776-konnectivity-ca\") pod \"konnectivity-agent-96mcv\" (UID: \"f94fa363-005b-460a-adea-49116e3b4776\") " pod="kube-system/konnectivity-agent-96mcv"
May 28 17:43:20.513080 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.511828    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b2eba6df-3e3e-46c8-b3c8-a1160503d901-ovnkube-script-lib\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.515624 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.515599    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz9ft\" (UniqueName: \"kubernetes.io/projected/90d40580-f219-42b1-9b78-9c97e23cf541-kube-api-access-zz9ft\") pod \"tuned-mk47q\" (UID: \"90d40580-f219-42b1-9b78-9c97e23cf541\") " pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.515710 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.515682    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5m5m\" (UniqueName: \"kubernetes.io/projected/6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d-kube-api-access-q5m5m\") pod \"aws-ebs-csi-driver-node-7w6kk\" (UID: \"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.515749 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.515735    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29bzf\" (UniqueName: \"kubernetes.io/projected/70ec7ff0-1089-4ca1-8238-da94d4b1c031-kube-api-access-29bzf\") pod \"node-resolver-5mv2t\" (UID: \"70ec7ff0-1089-4ca1-8238-da94d4b1c031\") " pod="openshift-dns/node-resolver-5mv2t"
May 28 17:43:20.522363 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.522340    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt8sh\" (UniqueName: \"kubernetes.io/projected/91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f-kube-api-access-qt8sh\") pod \"multus-27ntt\" (UID: \"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f\") " pod="openshift-multus/multus-27ntt"
May 28 17:43:20.522572 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.522555    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww24b\" (UniqueName: \"kubernetes.io/projected/538aa87a-ba23-4a27-80d9-73321fcf04a5-kube-api-access-ww24b\") pod \"iptables-alerter-xfjx8\" (UID: \"538aa87a-ba23-4a27-80d9-73321fcf04a5\") " pod="openshift-network-operator/iptables-alerter-xfjx8"
May 28 17:43:20.522801 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.522777    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qctrm\" (UniqueName: \"kubernetes.io/projected/b2eba6df-3e3e-46c8-b3c8-a1160503d901-kube-api-access-qctrm\") pod \"ovnkube-node-fsvt7\" (UID: \"b2eba6df-3e3e-46c8-b3c8-a1160503d901\") " pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.530078 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.530063    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk"
May 28 17:43:20.560894 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:20.560868    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6259ea4c_6fdb_4ab3_ae08_3deb4d76ff1d.slice/crio-d378c16057ba919ca5645e77a4ffa98379a05ede834069f6c75fc43f9388a60d WatchSource:0}: Error finding container d378c16057ba919ca5645e77a4ffa98379a05ede834069f6c75fc43f9388a60d: Status 404 returned error can't find the container with id d378c16057ba919ca5645e77a4ffa98379a05ede834069f6c75fc43f9388a60d
May 28 17:43:20.561082 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:20.561063    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7836682e045d93471bf2a629158ba14.slice/crio-05922e712cf5b2a72eb5a338755b0babf684e0268db106e515628aeca27b9e77 WatchSource:0}: Error finding container 05922e712cf5b2a72eb5a338755b0babf684e0268db106e515628aeca27b9e77: Status 404 returned error can't find the container with id 05922e712cf5b2a72eb5a338755b0babf684e0268db106e515628aeca27b9e77
May 28 17:43:20.565696 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.565677    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 17:43:20.571316 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:20.571295    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43f38dd408635a9aab60a231c225b6f8.slice/crio-f04f4b08dc669ccded9805b87a76a5e030ccb751c9beb9906cff0393ec027a0a WatchSource:0}: Error finding container f04f4b08dc669ccded9805b87a76a5e030ccb751c9beb9906cff0393ec027a0a: Status 404 returned error can't find the container with id f04f4b08dc669ccded9805b87a76a5e030ccb751c9beb9906cff0393ec027a0a
May 28 17:43:20.611673 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.611631    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-knh6s\" (UniqueName: \"kubernetes.io/projected/ec78263b-aee2-442b-8bc0-5f738c3d6832-kube-api-access-knh6s\") pod \"node-ca-46l5q\" (UID: \"ec78263b-aee2-442b-8bc0-5f738c3d6832\") " pod="openshift-image-registry/node-ca-46l5q"
May 28 17:43:20.611673 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.611672    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f454ed50-48e4-4202-88d3-1402e75de767-system-cni-dir\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.611922 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.611791    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f454ed50-48e4-4202-88d3-1402e75de767-cnibin\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.611922 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.611826    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f454ed50-48e4-4202-88d3-1402e75de767-system-cni-dir\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.611922 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.611850    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f454ed50-48e4-4202-88d3-1402e75de767-os-release\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.611922 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.611885    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs\") pod \"network-metrics-daemon-rh246\" (UID: \"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4\") " pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:20.611922 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.611899    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f454ed50-48e4-4202-88d3-1402e75de767-cnibin\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.612166 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.611949    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f454ed50-48e4-4202-88d3-1402e75de767-cni-binary-copy\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.612166 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.611991    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ec78263b-aee2-442b-8bc0-5f738c3d6832-serviceca\") pod \"node-ca-46l5q\" (UID: \"ec78263b-aee2-442b-8bc0-5f738c3d6832\") " pod="openshift-image-registry/node-ca-46l5q"
May 28 17:43:20.612166 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.611999    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f454ed50-48e4-4202-88d3-1402e75de767-os-release\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.612166 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:20.612007    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:43:20.612166 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.612035    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec78263b-aee2-442b-8bc0-5f738c3d6832-host\") pod \"node-ca-46l5q\" (UID: \"ec78263b-aee2-442b-8bc0-5f738c3d6832\") " pod="openshift-image-registry/node-ca-46l5q"
May 28 17:43:20.612166 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:20.612069    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs podName:4693da93-c00a-4dd7-8dd0-fb3be0c73ad4 nodeName:}" failed. No retries permitted until 2026-05-28 17:43:21.112044531 +0000 UTC m=+2.215740230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs") pod "network-metrics-daemon-rh246" (UID: "4693da93-c00a-4dd7-8dd0-fb3be0c73ad4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:43:20.612166 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.612077    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec78263b-aee2-442b-8bc0-5f738c3d6832-host\") pod \"node-ca-46l5q\" (UID: \"ec78263b-aee2-442b-8bc0-5f738c3d6832\") " pod="openshift-image-registry/node-ca-46l5q"
May 28 17:43:20.612166 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.612127    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bqvm6\" (UniqueName: \"kubernetes.io/projected/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-kube-api-access-bqvm6\") pod \"network-metrics-daemon-rh246\" (UID: \"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4\") " pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:20.612166 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.612152    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x62nf\" (UniqueName: \"kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf\") pod \"network-check-target-ch7tt\" (UID: \"0eec8ff8-0b7b-4d4d-a787-463843df5ef7\") " pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:20.612582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.612185    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f454ed50-48e4-4202-88d3-1402e75de767-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.612582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.612220    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f454ed50-48e4-4202-88d3-1402e75de767-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.612582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.612246    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f454ed50-48e4-4202-88d3-1402e75de767-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.612582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.612310    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l6956\" (UniqueName: \"kubernetes.io/projected/f454ed50-48e4-4202-88d3-1402e75de767-kube-api-access-l6956\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.612582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.612498    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ec78263b-aee2-442b-8bc0-5f738c3d6832-serviceca\") pod \"node-ca-46l5q\" (UID: \"ec78263b-aee2-442b-8bc0-5f738c3d6832\") " pod="openshift-image-registry/node-ca-46l5q"
May 28 17:43:20.612582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.612514    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f454ed50-48e4-4202-88d3-1402e75de767-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.612582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.612579    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f454ed50-48e4-4202-88d3-1402e75de767-cni-binary-copy\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.612965 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.612744    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f454ed50-48e4-4202-88d3-1402e75de767-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.612965 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.612777    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f454ed50-48e4-4202-88d3-1402e75de767-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.618302 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:20.618268    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:43:20.618302 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:20.618297    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:43:20.618302 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:20.618310    2560 projected.go:194] Error preparing data for projected volume kube-api-access-x62nf for pod openshift-network-diagnostics/network-check-target-ch7tt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:43:20.618497 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:20.618386    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf podName:0eec8ff8-0b7b-4d4d-a787-463843df5ef7 nodeName:}" failed. No retries permitted until 2026-05-28 17:43:21.118365613 +0000 UTC m=+2.222061324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-x62nf" (UniqueName: "kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf") pod "network-check-target-ch7tt" (UID: "0eec8ff8-0b7b-4d4d-a787-463843df5ef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:43:20.620624 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.620600    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knh6s\" (UniqueName: \"kubernetes.io/projected/ec78263b-aee2-442b-8bc0-5f738c3d6832-kube-api-access-knh6s\") pod \"node-ca-46l5q\" (UID: \"ec78263b-aee2-442b-8bc0-5f738c3d6832\") " pod="openshift-image-registry/node-ca-46l5q"
May 28 17:43:20.620624 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.620619    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqvm6\" (UniqueName: \"kubernetes.io/projected/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-kube-api-access-bqvm6\") pod \"network-metrics-daemon-rh246\" (UID: \"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4\") " pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:20.620755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.620731    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6956\" (UniqueName: \"kubernetes.io/projected/f454ed50-48e4-4202-88d3-1402e75de767-kube-api-access-l6956\") pod \"multus-additional-cni-plugins-k47r7\" (UID: \"f454ed50-48e4-4202-88d3-1402e75de767\") " pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.711366 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.711340    2560 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 17:43:20.720394 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.720306    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xfjx8"
May 28 17:43:20.727802 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:20.727770    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod538aa87a_ba23_4a27_80d9_73321fcf04a5.slice/crio-4379057c3b4e13e17c013faaf4cd1d2564c5a46acf877ceb1d49ca1fd70e5a07 WatchSource:0}: Error finding container 4379057c3b4e13e17c013faaf4cd1d2564c5a46acf877ceb1d49ca1fd70e5a07: Status 404 returned error can't find the container with id 4379057c3b4e13e17c013faaf4cd1d2564c5a46acf877ceb1d49ca1fd70e5a07
May 28 17:43:20.736704 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.736669    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:20.743732 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:20.743699    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2eba6df_3e3e_46c8_b3c8_a1160503d901.slice/crio-e620400226ea26b725041defe07fb5d7a099937051f411f09f56a3864b2d2d9c WatchSource:0}: Error finding container e620400226ea26b725041defe07fb5d7a099937051f411f09f56a3864b2d2d9c: Status 404 returned error can't find the container with id e620400226ea26b725041defe07fb5d7a099937051f411f09f56a3864b2d2d9c
May 28 17:43:20.773140 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.773092    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-96mcv"
May 28 17:43:20.773140 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.773115    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mk47q"
May 28 17:43:20.781905 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:20.781871    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90d40580_f219_42b1_9b78_9c97e23cf541.slice/crio-d137648667ac496cc879b534e7187b95fdc3bd0f11ef62835933d708366a713d WatchSource:0}: Error finding container d137648667ac496cc879b534e7187b95fdc3bd0f11ef62835933d708366a713d: Status 404 returned error can't find the container with id d137648667ac496cc879b534e7187b95fdc3bd0f11ef62835933d708366a713d
May 28 17:43:20.782564 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:20.782536    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf94fa363_005b_460a_adea_49116e3b4776.slice/crio-84710f29981d814ec572f156c2f6ec9ee085b763bdc310f42f77442218b093a6 WatchSource:0}: Error finding container 84710f29981d814ec572f156c2f6ec9ee085b763bdc310f42f77442218b093a6: Status 404 returned error can't find the container with id 84710f29981d814ec572f156c2f6ec9ee085b763bdc310f42f77442218b093a6
May 28 17:43:20.795707 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.795678    2560 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 17:43:20.797438 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.797418    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5mv2t"
May 28 17:43:20.803701 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:20.803671    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70ec7ff0_1089_4ca1_8238_da94d4b1c031.slice/crio-85c84839dcafd9d82322ae4d4a63128cbfdf3aaa6e2a8568dbecf65fe8c98a0f WatchSource:0}: Error finding container 85c84839dcafd9d82322ae4d4a63128cbfdf3aaa6e2a8568dbecf65fe8c98a0f: Status 404 returned error can't find the container with id 85c84839dcafd9d82322ae4d4a63128cbfdf3aaa6e2a8568dbecf65fe8c98a0f
May 28 17:43:20.812769 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.812733    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-27ntt"
May 28 17:43:20.853898 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.853866    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-46l5q"
May 28 17:43:20.858617 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:20.858594    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k47r7"
May 28 17:43:20.861421 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:20.861392    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec78263b_aee2_442b_8bc0_5f738c3d6832.slice/crio-32beaac0abea78a7331a3df995846c7f8f09639f981e1d53df697e680af762d4 WatchSource:0}: Error finding container 32beaac0abea78a7331a3df995846c7f8f09639f981e1d53df697e680af762d4: Status 404 returned error can't find the container with id 32beaac0abea78a7331a3df995846c7f8f09639f981e1d53df697e680af762d4
May 28 17:43:20.866906 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:43:20.866878    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf454ed50_48e4_4202_88d3_1402e75de767.slice/crio-9b3cd87b9e9733fc6739409723883c714a71f80b2def7fa6b6572f5de06166be WatchSource:0}: Error finding container 9b3cd87b9e9733fc6739409723883c714a71f80b2def7fa6b6572f5de06166be: Status 404 returned error can't find the container with id 9b3cd87b9e9733fc6739409723883c714a71f80b2def7fa6b6572f5de06166be
May 28 17:43:21.116185 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:21.116148    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs\") pod \"network-metrics-daemon-rh246\" (UID: \"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4\") " pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:21.116359 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:21.116287    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:43:21.116359 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:21.116347    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs podName:4693da93-c00a-4dd7-8dd0-fb3be0c73ad4 nodeName:}" failed. No retries permitted until 2026-05-28 17:43:22.11632908 +0000 UTC m=+3.220024790 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs") pod "network-metrics-daemon-rh246" (UID: "4693da93-c00a-4dd7-8dd0-fb3be0c73ad4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:43:21.217645 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:21.217608    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x62nf\" (UniqueName: \"kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf\") pod \"network-check-target-ch7tt\" (UID: \"0eec8ff8-0b7b-4d4d-a787-463843df5ef7\") " pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:21.217830 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:21.217774    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:43:21.217830 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:21.217791    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:43:21.217830 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:21.217804    2560 projected.go:194] Error preparing data for projected volume kube-api-access-x62nf for pod openshift-network-diagnostics/network-check-target-ch7tt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:43:21.218014 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:21.217858    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf podName:0eec8ff8-0b7b-4d4d-a787-463843df5ef7 nodeName:}" failed. No retries permitted until 2026-05-28 17:43:22.217840081 +0000 UTC m=+3.321535783 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-x62nf" (UniqueName: "kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf") pod "network-check-target-ch7tt" (UID: "0eec8ff8-0b7b-4d4d-a787-463843df5ef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:43:21.440051 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:21.439961    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 17:38:20 +0000 UTC" deadline="2027-12-25 12:10:08.82755844 +0000 UTC"
May 28 17:43:21.440051 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:21.440007    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13818h26m47.387557135s"
May 28 17:43:21.543831 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:21.543768    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-41.ec2.internal" event={"ID":"a7836682e045d93471bf2a629158ba14","Type":"ContainerStarted","Data":"05922e712cf5b2a72eb5a338755b0babf684e0268db106e515628aeca27b9e77"}
May 28 17:43:21.550052 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:21.550019    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk" event={"ID":"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d","Type":"ContainerStarted","Data":"d378c16057ba919ca5645e77a4ffa98379a05ede834069f6c75fc43f9388a60d"}
May 28 17:43:21.565880 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:21.565819    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5mv2t" event={"ID":"70ec7ff0-1089-4ca1-8238-da94d4b1c031","Type":"ContainerStarted","Data":"85c84839dcafd9d82322ae4d4a63128cbfdf3aaa6e2a8568dbecf65fe8c98a0f"}
May 28 17:43:21.586290 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:21.586253    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-96mcv" event={"ID":"f94fa363-005b-460a-adea-49116e3b4776","Type":"ContainerStarted","Data":"84710f29981d814ec572f156c2f6ec9ee085b763bdc310f42f77442218b093a6"}
May 28 17:43:21.589766 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:21.589736    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mk47q" event={"ID":"90d40580-f219-42b1-9b78-9c97e23cf541","Type":"ContainerStarted","Data":"d137648667ac496cc879b534e7187b95fdc3bd0f11ef62835933d708366a713d"}
May 28 17:43:21.626819 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:21.626784    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k47r7" event={"ID":"f454ed50-48e4-4202-88d3-1402e75de767","Type":"ContainerStarted","Data":"9b3cd87b9e9733fc6739409723883c714a71f80b2def7fa6b6572f5de06166be"}
May 28 17:43:21.651520 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:21.651380    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-46l5q" event={"ID":"ec78263b-aee2-442b-8bc0-5f738c3d6832","Type":"ContainerStarted","Data":"32beaac0abea78a7331a3df995846c7f8f09639f981e1d53df697e680af762d4"}
May 28 17:43:21.664076 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:21.664042    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-27ntt" event={"ID":"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f","Type":"ContainerStarted","Data":"e3519689407a28717531097dddb51bfff91fc4c9cbc9f7ca01a8f4fd706b8a5a"}
May 28 17:43:21.671997 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:21.671970    2560 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 17:43:21.677412 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:21.677385    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7" event={"ID":"b2eba6df-3e3e-46c8-b3c8-a1160503d901","Type":"ContainerStarted","Data":"e620400226ea26b725041defe07fb5d7a099937051f411f09f56a3864b2d2d9c"}
May 28 17:43:21.688804 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:21.688774    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xfjx8" event={"ID":"538aa87a-ba23-4a27-80d9-73321fcf04a5","Type":"ContainerStarted","Data":"4379057c3b4e13e17c013faaf4cd1d2564c5a46acf877ceb1d49ca1fd70e5a07"}
May 28 17:43:21.703480 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:21.703405    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal" event={"ID":"43f38dd408635a9aab60a231c225b6f8","Type":"ContainerStarted","Data":"f04f4b08dc669ccded9805b87a76a5e030ccb751c9beb9906cff0393ec027a0a"}
May 28 17:43:22.125797 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:22.125753    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs\") pod \"network-metrics-daemon-rh246\" (UID: \"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4\") " pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:22.126036 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:22.125980    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:43:22.126115 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:22.126053    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs podName:4693da93-c00a-4dd7-8dd0-fb3be0c73ad4 nodeName:}" failed. No retries permitted until 2026-05-28 17:43:24.12603232 +0000 UTC m=+5.229728039 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs") pod "network-metrics-daemon-rh246" (UID: "4693da93-c00a-4dd7-8dd0-fb3be0c73ad4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:43:22.226545 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:22.226506    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x62nf\" (UniqueName: \"kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf\") pod \"network-check-target-ch7tt\" (UID: \"0eec8ff8-0b7b-4d4d-a787-463843df5ef7\") " pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:22.226741 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:22.226711    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:43:22.226741 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:22.226740    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:43:22.226843 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:22.226754    2560 projected.go:194] Error preparing data for projected volume kube-api-access-x62nf for pod openshift-network-diagnostics/network-check-target-ch7tt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:43:22.226843 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:22.226810    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf podName:0eec8ff8-0b7b-4d4d-a787-463843df5ef7 nodeName:}" failed. No retries permitted until 2026-05-28 17:43:24.226792496 +0000 UTC m=+5.330488204 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-x62nf" (UniqueName: "kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf") pod "network-check-target-ch7tt" (UID: "0eec8ff8-0b7b-4d4d-a787-463843df5ef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:43:22.440432 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:22.440312    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 17:38:20 +0000 UTC" deadline="2028-01-13 02:12:52.721877414 +0000 UTC"
May 28 17:43:22.440432 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:22.440351    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14264h29m30.281530164s"
May 28 17:43:22.515462 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:22.514968    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:22.515462 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:22.515040    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:22.515462 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:22.515107    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:22.515462 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:22.515277    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:24.141076 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:24.141035    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs\") pod \"network-metrics-daemon-rh246\" (UID: \"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4\") " pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:24.141569 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:24.141190    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:43:24.141569 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:24.141261    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs podName:4693da93-c00a-4dd7-8dd0-fb3be0c73ad4 nodeName:}" failed. No retries permitted until 2026-05-28 17:43:28.141241228 +0000 UTC m=+9.244936930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs") pod "network-metrics-daemon-rh246" (UID: "4693da93-c00a-4dd7-8dd0-fb3be0c73ad4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:43:24.241615 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:24.241582    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x62nf\" (UniqueName: \"kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf\") pod \"network-check-target-ch7tt\" (UID: \"0eec8ff8-0b7b-4d4d-a787-463843df5ef7\") " pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:24.241787 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:24.241764    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:43:24.241787 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:24.241783    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:43:24.241908 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:24.241796    2560 projected.go:194] Error preparing data for projected volume kube-api-access-x62nf for pod openshift-network-diagnostics/network-check-target-ch7tt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:43:24.241908 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:24.241857    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf podName:0eec8ff8-0b7b-4d4d-a787-463843df5ef7 nodeName:}" failed. No retries permitted until 2026-05-28 17:43:28.241836278 +0000 UTC m=+9.345531991 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-x62nf" (UniqueName: "kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf") pod "network-check-target-ch7tt" (UID: "0eec8ff8-0b7b-4d4d-a787-463843df5ef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:43:24.513889 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:24.513812    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:24.514056 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:24.513979    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:24.514425 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:24.514366    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:24.514483 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:24.514460    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:26.514194 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:26.514157    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:26.514676 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:26.514308    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:26.514676 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:26.514404    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:26.514676 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:26.514498    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:28.173572 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:28.173517    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs\") pod \"network-metrics-daemon-rh246\" (UID: \"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4\") " pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:28.174009 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:28.173701    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:43:28.174009 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:28.173753    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs podName:4693da93-c00a-4dd7-8dd0-fb3be0c73ad4 nodeName:}" failed. No retries permitted until 2026-05-28 17:43:36.173739488 +0000 UTC m=+17.277435188 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs") pod "network-metrics-daemon-rh246" (UID: "4693da93-c00a-4dd7-8dd0-fb3be0c73ad4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:43:28.274770 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:28.274735    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x62nf\" (UniqueName: \"kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf\") pod \"network-check-target-ch7tt\" (UID: \"0eec8ff8-0b7b-4d4d-a787-463843df5ef7\") " pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:28.274950 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:28.274892    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:43:28.274950 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:28.274909    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:43:28.274950 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:28.274921    2560 projected.go:194] Error preparing data for projected volume kube-api-access-x62nf for pod openshift-network-diagnostics/network-check-target-ch7tt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:43:28.275100 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:28.274993    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf podName:0eec8ff8-0b7b-4d4d-a787-463843df5ef7 nodeName:}" failed. No retries permitted until 2026-05-28 17:43:36.274973269 +0000 UTC m=+17.378668975 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-x62nf" (UniqueName: "kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf") pod "network-check-target-ch7tt" (UID: "0eec8ff8-0b7b-4d4d-a787-463843df5ef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:43:28.513563 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:28.513479    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:28.513729 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:28.513599    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:28.513729 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:28.513655    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:28.513819 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:28.513745    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:30.514492 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:30.513866    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:30.514492 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:30.514044    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:30.514492 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:30.514369    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:30.514492 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:30.514449    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:32.514031 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:32.513995    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:32.514418 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:32.513996    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:32.514418 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:32.514116    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:32.514418 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:32.514196    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:34.383533 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:34.383493    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-hgbhs"]
May 28 17:43:34.407107 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:34.407073    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:34.407266 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:34.407174    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgbhs" podUID="a8061064-d0c1-4eea-bb4c-2c237c83b89c"
May 28 17:43:34.513248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:34.513211    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:34.513426 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:34.513208    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:34.513426 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:34.513334    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:34.513515 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:34.513445    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:34.527666 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:34.527628    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret\") pod \"global-pull-secret-syncer-hgbhs\" (UID: \"a8061064-d0c1-4eea-bb4c-2c237c83b89c\") " pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:34.527832 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:34.527720    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a8061064-d0c1-4eea-bb4c-2c237c83b89c-dbus\") pod \"global-pull-secret-syncer-hgbhs\" (UID: \"a8061064-d0c1-4eea-bb4c-2c237c83b89c\") " pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:34.527832 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:34.527770    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a8061064-d0c1-4eea-bb4c-2c237c83b89c-kubelet-config\") pod \"global-pull-secret-syncer-hgbhs\" (UID: \"a8061064-d0c1-4eea-bb4c-2c237c83b89c\") " pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:34.628671 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:34.628631    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret\") pod \"global-pull-secret-syncer-hgbhs\" (UID: \"a8061064-d0c1-4eea-bb4c-2c237c83b89c\") " pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:34.628671 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:34.628680    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a8061064-d0c1-4eea-bb4c-2c237c83b89c-dbus\") pod \"global-pull-secret-syncer-hgbhs\" (UID: \"a8061064-d0c1-4eea-bb4c-2c237c83b89c\") " pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:34.629010 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:34.628728    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a8061064-d0c1-4eea-bb4c-2c237c83b89c-kubelet-config\") pod \"global-pull-secret-syncer-hgbhs\" (UID: \"a8061064-d0c1-4eea-bb4c-2c237c83b89c\") " pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:34.629010 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:34.628797    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 17:43:34.629010 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:34.628818    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a8061064-d0c1-4eea-bb4c-2c237c83b89c-kubelet-config\") pod \"global-pull-secret-syncer-hgbhs\" (UID: \"a8061064-d0c1-4eea-bb4c-2c237c83b89c\") " pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:34.629010 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:34.628863    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret podName:a8061064-d0c1-4eea-bb4c-2c237c83b89c nodeName:}" failed. No retries permitted until 2026-05-28 17:43:35.128843419 +0000 UTC m=+16.232539136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret") pod "global-pull-secret-syncer-hgbhs" (UID: "a8061064-d0c1-4eea-bb4c-2c237c83b89c") : object "kube-system"/"original-pull-secret" not registered
May 28 17:43:34.629010 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:34.628893    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a8061064-d0c1-4eea-bb4c-2c237c83b89c-dbus\") pod \"global-pull-secret-syncer-hgbhs\" (UID: \"a8061064-d0c1-4eea-bb4c-2c237c83b89c\") " pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:35.132204 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:35.132159    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret\") pod \"global-pull-secret-syncer-hgbhs\" (UID: \"a8061064-d0c1-4eea-bb4c-2c237c83b89c\") " pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:35.132409 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:35.132311    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 17:43:35.132409 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:35.132388    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret podName:a8061064-d0c1-4eea-bb4c-2c237c83b89c nodeName:}" failed. No retries permitted until 2026-05-28 17:43:36.132371994 +0000 UTC m=+17.236067691 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret") pod "global-pull-secret-syncer-hgbhs" (UID: "a8061064-d0c1-4eea-bb4c-2c237c83b89c") : object "kube-system"/"original-pull-secret" not registered
May 28 17:43:35.513304 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:35.513235    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:35.513737 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:35.513331    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgbhs" podUID="a8061064-d0c1-4eea-bb4c-2c237c83b89c"
May 28 17:43:36.137944 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:36.137900    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret\") pod \"global-pull-secret-syncer-hgbhs\" (UID: \"a8061064-d0c1-4eea-bb4c-2c237c83b89c\") " pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:36.138251 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:36.138062    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 17:43:36.138251 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:36.138140    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret podName:a8061064-d0c1-4eea-bb4c-2c237c83b89c nodeName:}" failed. No retries permitted until 2026-05-28 17:43:38.138119548 +0000 UTC m=+19.241815258 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret") pod "global-pull-secret-syncer-hgbhs" (UID: "a8061064-d0c1-4eea-bb4c-2c237c83b89c") : object "kube-system"/"original-pull-secret" not registered
May 28 17:43:36.238486 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:36.238449    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs\") pod \"network-metrics-daemon-rh246\" (UID: \"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4\") " pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:36.238642 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:36.238606    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:43:36.238689 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:36.238682    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs podName:4693da93-c00a-4dd7-8dd0-fb3be0c73ad4 nodeName:}" failed. No retries permitted until 2026-05-28 17:43:52.238658279 +0000 UTC m=+33.342353980 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs") pod "network-metrics-daemon-rh246" (UID: "4693da93-c00a-4dd7-8dd0-fb3be0c73ad4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:43:36.339250 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:36.339204    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x62nf\" (UniqueName: \"kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf\") pod \"network-check-target-ch7tt\" (UID: \"0eec8ff8-0b7b-4d4d-a787-463843df5ef7\") " pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:36.339430 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:36.339400    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:43:36.339430 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:36.339418    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:43:36.339430 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:36.339431    2560 projected.go:194] Error preparing data for projected volume kube-api-access-x62nf for pod openshift-network-diagnostics/network-check-target-ch7tt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:43:36.339576 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:36.339491    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf podName:0eec8ff8-0b7b-4d4d-a787-463843df5ef7 nodeName:}" failed. No retries permitted until 2026-05-28 17:43:52.339473536 +0000 UTC m=+33.443169254 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-x62nf" (UniqueName: "kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf") pod "network-check-target-ch7tt" (UID: "0eec8ff8-0b7b-4d4d-a787-463843df5ef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:43:36.514482 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:36.513907    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:36.514482 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:36.513980    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:36.514482 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:36.514063    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:36.514482 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:36.514200    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:37.513858 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:37.513606    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:37.514049 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:37.513973    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgbhs" podUID="a8061064-d0c1-4eea-bb4c-2c237c83b89c"
May 28 17:43:38.155077 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:38.155025    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret\") pod \"global-pull-secret-syncer-hgbhs\" (UID: \"a8061064-d0c1-4eea-bb4c-2c237c83b89c\") " pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:38.155489 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:38.155209    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 17:43:38.155489 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:38.155261    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret podName:a8061064-d0c1-4eea-bb4c-2c237c83b89c nodeName:}" failed. No retries permitted until 2026-05-28 17:43:42.155247267 +0000 UTC m=+23.258942966 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret") pod "global-pull-secret-syncer-hgbhs" (UID: "a8061064-d0c1-4eea-bb4c-2c237c83b89c") : object "kube-system"/"original-pull-secret" not registered
May 28 17:43:38.513292 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:38.513203    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:38.513462 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:38.513205    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:38.513462 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:38.513335    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:38.513462 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:38.513428    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:39.514294 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:39.514257    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:39.514754 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:39.514354    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgbhs" podUID="a8061064-d0c1-4eea-bb4c-2c237c83b89c"
May 28 17:43:40.513873 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:40.513834    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:40.514087 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:40.513839    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:40.514087 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:40.513987    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:40.514087 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:40.514018    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:41.513615 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.513441    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:41.514259 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:41.513684    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgbhs" podUID="a8061064-d0c1-4eea-bb4c-2c237c83b89c"
May 28 17:43:41.743004 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.742970    2560 generic.go:358] "Generic (PLEG): container finished" podID="f454ed50-48e4-4202-88d3-1402e75de767" containerID="a76c8619683c450f00d1ab08be51bd0ce0fd22882fc716b59eae546621e1158f" exitCode=0
May 28 17:43:41.743181 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.743044    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k47r7" event={"ID":"f454ed50-48e4-4202-88d3-1402e75de767","Type":"ContainerDied","Data":"a76c8619683c450f00d1ab08be51bd0ce0fd22882fc716b59eae546621e1158f"}
May 28 17:43:41.744459 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.744383    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-46l5q" event={"ID":"ec78263b-aee2-442b-8bc0-5f738c3d6832","Type":"ContainerStarted","Data":"a3c4332924628b9f5132ceadd1e10da51159a1d63425877ec45a776e18204f00"}
May 28 17:43:41.745800 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.745770    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-27ntt" event={"ID":"91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f","Type":"ContainerStarted","Data":"9bf039cf73f7f11fa3a4207ca70a95ea5089188a8eb470ad1acedfef385624fe"}
May 28 17:43:41.748460 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.748434    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7" event={"ID":"b2eba6df-3e3e-46c8-b3c8-a1160503d901","Type":"ContainerStarted","Data":"2c2ab0acdca679af9db84fc86580d85588694ee249eb4a17ac4181af847dafdb"}
May 28 17:43:41.748562 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.748474    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7" event={"ID":"b2eba6df-3e3e-46c8-b3c8-a1160503d901","Type":"ContainerStarted","Data":"c3d19ff829bc32bc437631ef1b3f86351634d105f0cbcea88894efeb9bb4305c"}
May 28 17:43:41.748562 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.748487    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7" event={"ID":"b2eba6df-3e3e-46c8-b3c8-a1160503d901","Type":"ContainerStarted","Data":"878cef76e458fbc6040e55b24186756c634679077cdcd4b0acb8d50abe84db3d"}
May 28 17:43:41.748562 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.748499    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7" event={"ID":"b2eba6df-3e3e-46c8-b3c8-a1160503d901","Type":"ContainerStarted","Data":"ffbe06c31ca078a5d9cac758ac0e24bbdebf4e8fb951c3cf567ef10c10911570"}
May 28 17:43:41.748562 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.748524    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7" event={"ID":"b2eba6df-3e3e-46c8-b3c8-a1160503d901","Type":"ContainerStarted","Data":"fcab81a150ba6eb7d2669a63e5bcc3a052fe86751085327c16e425e1e0760d5a"}
May 28 17:43:41.748562 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.748535    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7" event={"ID":"b2eba6df-3e3e-46c8-b3c8-a1160503d901","Type":"ContainerStarted","Data":"ca07c2a5eb9f4f2eb83e57ce7a59d7b9cdc32a12f2a2b56c7ddba76e8d4c645c"}
May 28 17:43:41.749718 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.749697    2560 generic.go:358] "Generic (PLEG): container finished" podID="43f38dd408635a9aab60a231c225b6f8" containerID="c7a40f86159e36bdb71c0a16ee3719662a5026cc535ca0816f1e9abf7cc94123" exitCode=0
May 28 17:43:41.749811 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.749760    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal" event={"ID":"43f38dd408635a9aab60a231c225b6f8","Type":"ContainerDied","Data":"c7a40f86159e36bdb71c0a16ee3719662a5026cc535ca0816f1e9abf7cc94123"}
May 28 17:43:41.751012 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.750988    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-41.ec2.internal" event={"ID":"a7836682e045d93471bf2a629158ba14","Type":"ContainerStarted","Data":"be29a8229c00d4437124190c94c6feca30c3589b760bb48534e5bb6a757da20d"}
May 28 17:43:41.752138 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.752114    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk" event={"ID":"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d","Type":"ContainerStarted","Data":"dfccde02c67caa8830bf3af816e0abdf34437d55a1200ea76f91f5450f97d7ef"}
May 28 17:43:41.753323 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.753301    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5mv2t" event={"ID":"70ec7ff0-1089-4ca1-8238-da94d4b1c031","Type":"ContainerStarted","Data":"ba38a71a7bd65156ba9b9b2208487bc54ed3e001974467b882a257144b4d2308"}
May 28 17:43:41.754577 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.754554    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-96mcv" event={"ID":"f94fa363-005b-460a-adea-49116e3b4776","Type":"ContainerStarted","Data":"94a9c57b18b09b31d6df711e9fb3efc5788ea582a5945b5b6eed01fbe828afef"}
May 28 17:43:41.755766 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.755747    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mk47q" event={"ID":"90d40580-f219-42b1-9b78-9c97e23cf541","Type":"ContainerStarted","Data":"b70253d9deb45a43381c3ec0669b2d04f43a85f89b08ce6dbac2a877ae99da04"}
May 28 17:43:41.819241 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.819050    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-96mcv" podStartSLOduration=2.783997778 podStartE2EDuration="22.819035546s" podCreationTimestamp="2026-05-28 17:43:19 +0000 UTC" firstStartedPulling="2026-05-28 17:43:20.784281003 +0000 UTC m=+1.887976699" lastFinishedPulling="2026-05-28 17:43:40.819318754 +0000 UTC m=+21.923014467" observedRunningTime="2026-05-28 17:43:41.8007402 +0000 UTC m=+22.904436105" watchObservedRunningTime="2026-05-28 17:43:41.819035546 +0000 UTC m=+22.922731268"
May 28 17:43:41.819428 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.819360    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-27ntt" podStartSLOduration=2.812572491 podStartE2EDuration="22.819353865s" podCreationTimestamp="2026-05-28 17:43:19 +0000 UTC" firstStartedPulling="2026-05-28 17:43:20.822081224 +0000 UTC m=+1.925776921" lastFinishedPulling="2026-05-28 17:43:40.8288626 +0000 UTC m=+21.932558295" observedRunningTime="2026-05-28 17:43:41.818611204 +0000 UTC m=+22.922306916" watchObservedRunningTime="2026-05-28 17:43:41.819353865 +0000 UTC m=+22.923049583"
May 28 17:43:41.831059 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.831017    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-46l5q" podStartSLOduration=11.331029819 podStartE2EDuration="22.831003929s" podCreationTimestamp="2026-05-28 17:43:19 +0000 UTC" firstStartedPulling="2026-05-28 17:43:20.863478205 +0000 UTC m=+1.967173902" lastFinishedPulling="2026-05-28 17:43:32.363452304 +0000 UTC m=+13.467148012" observedRunningTime="2026-05-28 17:43:41.830845802 +0000 UTC m=+22.934541522" watchObservedRunningTime="2026-05-28 17:43:41.831003929 +0000 UTC m=+22.934699644"
May 28 17:43:41.857502 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.857403    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-41.ec2.internal" podStartSLOduration=21.857387156 podStartE2EDuration="21.857387156s" podCreationTimestamp="2026-05-28 17:43:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:43:41.843053855 +0000 UTC m=+22.946749586" watchObservedRunningTime="2026-05-28 17:43:41.857387156 +0000 UTC m=+22.961082874"
May 28 17:43:41.857903 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.857878    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mk47q" podStartSLOduration=2.820876769 podStartE2EDuration="22.857870616s" podCreationTimestamp="2026-05-28 17:43:19 +0000 UTC" firstStartedPulling="2026-05-28 17:43:20.783731496 +0000 UTC m=+1.887427207" lastFinishedPulling="2026-05-28 17:43:40.820725344 +0000 UTC m=+21.924421054" observedRunningTime="2026-05-28 17:43:41.857362417 +0000 UTC m=+22.961058135" watchObservedRunningTime="2026-05-28 17:43:41.857870616 +0000 UTC m=+22.961566377"
May 28 17:43:41.873267 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:41.873223    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-5mv2t" podStartSLOduration=2.858890467 podStartE2EDuration="22.873208773s" podCreationTimestamp="2026-05-28 17:43:19 +0000 UTC" firstStartedPulling="2026-05-28 17:43:20.805306925 +0000 UTC m=+1.909002623" lastFinishedPulling="2026-05-28 17:43:40.819625232 +0000 UTC m=+21.923320929" observedRunningTime="2026-05-28 17:43:41.872689781 +0000 UTC m=+22.976385501" watchObservedRunningTime="2026-05-28 17:43:41.873208773 +0000 UTC m=+22.976904491"
May 28 17:43:42.185397 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:42.185368    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret\") pod \"global-pull-secret-syncer-hgbhs\" (UID: \"a8061064-d0c1-4eea-bb4c-2c237c83b89c\") " pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:42.185567 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:42.185479    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 17:43:42.185567 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:42.185544    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret podName:a8061064-d0c1-4eea-bb4c-2c237c83b89c nodeName:}" failed. No retries permitted until 2026-05-28 17:43:50.185526545 +0000 UTC m=+31.289222244 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret") pod "global-pull-secret-syncer-hgbhs" (UID: "a8061064-d0c1-4eea-bb4c-2c237c83b89c") : object "kube-system"/"original-pull-secret" not registered
May 28 17:43:42.485979 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:42.485917    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 28 17:43:42.488815 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:42.488713    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T17:43:42.48596068Z","UUID":"c0f6fc63-7a12-4f30-9cc3-0843abffd1f9","Handler":null,"Name":"","Endpoint":""}
May 28 17:43:42.491178 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:42.491150    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 28 17:43:42.491354 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:42.491190    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 28 17:43:42.514259 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:42.514224    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:42.514665 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:42.514357    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:42.514665 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:42.514411    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:42.514665 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:42.514506    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:42.758883 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:42.758790    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xfjx8" event={"ID":"538aa87a-ba23-4a27-80d9-73321fcf04a5","Type":"ContainerStarted","Data":"1772d6928c7cadc525c3163343b70c666f4c76b2ed78652896f717e5e82792b3"}
May 28 17:43:42.760535 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:42.760503    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal" event={"ID":"43f38dd408635a9aab60a231c225b6f8","Type":"ContainerStarted","Data":"d378e499e64cc48b460c59c040f2eaa17fe4b4ac34c3d430529aac091ac8763d"}
May 28 17:43:42.762703 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:42.762678    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk" event={"ID":"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d","Type":"ContainerStarted","Data":"9efe2e428888fd1dbabb2727dc26313d427fce37a172b2fea65bdcdd726cdee2"}
May 28 17:43:42.772427 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:42.772388    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xfjx8" podStartSLOduration=3.683039945 podStartE2EDuration="23.772372363s" podCreationTimestamp="2026-05-28 17:43:19 +0000 UTC" firstStartedPulling="2026-05-28 17:43:20.72998488 +0000 UTC m=+1.833680577" lastFinishedPulling="2026-05-28 17:43:40.819317285 +0000 UTC m=+21.923012995" observedRunningTime="2026-05-28 17:43:42.771997666 +0000 UTC m=+23.875693385" watchObservedRunningTime="2026-05-28 17:43:42.772372363 +0000 UTC m=+23.876068079"
May 28 17:43:42.785862 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:42.785827    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-41.ec2.internal" podStartSLOduration=22.785815776 podStartE2EDuration="22.785815776s" podCreationTimestamp="2026-05-28 17:43:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:43:42.785727647 +0000 UTC m=+23.889423365" watchObservedRunningTime="2026-05-28 17:43:42.785815776 +0000 UTC m=+23.889511494"
May 28 17:43:42.789944 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:42.789908    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-96mcv"
May 28 17:43:42.790638 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:42.790621    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-96mcv"
May 28 17:43:43.516990 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:43.516959    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:43.517459 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:43.517082    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgbhs" podUID="a8061064-d0c1-4eea-bb4c-2c237c83b89c"
May 28 17:43:44.513487 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:44.513457    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:44.513665 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:44.513457    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:44.513665 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:44.513598    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:44.513665 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:44.513629    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:44.769485 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:44.769403    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7" event={"ID":"b2eba6df-3e3e-46c8-b3c8-a1160503d901","Type":"ContainerStarted","Data":"9d1e037ca57ea61d4fbef0be16b80e75e0bc032d69f132f474cd85d777dfcba3"}
May 28 17:43:44.771553 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:44.771532    2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 28 17:43:44.772015 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:44.771990    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk" event={"ID":"6259ea4c-6fdb-4ab3-ae08-3deb4d76ff1d","Type":"ContainerStarted","Data":"b0ae3d194b9ecb3023cce51e7214c3282aef04bb0d6ccbbfad726598850a9084"}
May 28 17:43:44.788320 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:44.788271    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7w6kk" podStartSLOduration=2.689835037 podStartE2EDuration="25.78825681s" podCreationTimestamp="2026-05-28 17:43:19 +0000 UTC" firstStartedPulling="2026-05-28 17:43:20.565947614 +0000 UTC m=+1.669643320" lastFinishedPulling="2026-05-28 17:43:43.664369395 +0000 UTC m=+24.768065093" observedRunningTime="2026-05-28 17:43:44.787379412 +0000 UTC m=+25.891075126" watchObservedRunningTime="2026-05-28 17:43:44.78825681 +0000 UTC m=+25.891952529"
May 28 17:43:45.513579 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:45.513543    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:45.513744 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:45.513669    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgbhs" podUID="a8061064-d0c1-4eea-bb4c-2c237c83b89c"
May 28 17:43:46.513397 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:46.513208    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:46.514148 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:46.513209    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:46.514148 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:46.513506    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:46.514148 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:46.513541    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:46.777876 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:46.777836    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7" event={"ID":"b2eba6df-3e3e-46c8-b3c8-a1160503d901","Type":"ContainerStarted","Data":"ffe1951f00508ffa0c0ae3f515647887c65214d4f3fa66ee4488a2a55f0c7917"}
May 28 17:43:46.778219 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:46.778156    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:46.779477 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:46.779453    2560 generic.go:358] "Generic (PLEG): container finished" podID="f454ed50-48e4-4202-88d3-1402e75de767" containerID="8f00ef1bafd776c9b2e95ece0f2fb2bea555a9e2f32578661ff9ebbb65260d47" exitCode=0
May 28 17:43:46.779583 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:46.779489    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k47r7" event={"ID":"f454ed50-48e4-4202-88d3-1402e75de767","Type":"ContainerDied","Data":"8f00ef1bafd776c9b2e95ece0f2fb2bea555a9e2f32578661ff9ebbb65260d47"}
May 28 17:43:46.794847 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:46.794824    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:46.802082 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:46.802042    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7" podStartSLOduration=7.587456603 podStartE2EDuration="27.802030246s" podCreationTimestamp="2026-05-28 17:43:19 +0000 UTC" firstStartedPulling="2026-05-28 17:43:20.745485308 +0000 UTC m=+1.849181004" lastFinishedPulling="2026-05-28 17:43:40.960058951 +0000 UTC m=+22.063754647" observedRunningTime="2026-05-28 17:43:46.800766699 +0000 UTC m=+27.904462451" watchObservedRunningTime="2026-05-28 17:43:46.802030246 +0000 UTC m=+27.905725964"
May 28 17:43:47.513889 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:47.513864    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:47.514552 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:47.513991    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgbhs" podUID="a8061064-d0c1-4eea-bb4c-2c237c83b89c"
May 28 17:43:47.782253 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:47.782215    2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 28 17:43:47.782755 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:47.782697    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:47.801781 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:47.801744    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:48.051573 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:48.051127    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hgbhs"]
May 28 17:43:48.051725 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:48.051659    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:48.051792 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:48.051772    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgbhs" podUID="a8061064-d0c1-4eea-bb4c-2c237c83b89c"
May 28 17:43:48.054609 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:48.054583    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ch7tt"]
May 28 17:43:48.054740 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:48.054621    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rh246"]
May 28 17:43:48.054740 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:48.054730    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:48.054860 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:48.054834    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:48.054995 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:48.054973    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:48.055104 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:48.055082    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:48.786014 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:48.785986    2560 generic.go:358] "Generic (PLEG): container finished" podID="f454ed50-48e4-4202-88d3-1402e75de767" containerID="2b768f833e7a6bc589b28a5655a2b09598956d72c496f9471773565cca772cbf" exitCode=0
May 28 17:43:48.786438 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:48.786071    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k47r7" event={"ID":"f454ed50-48e4-4202-88d3-1402e75de767","Type":"ContainerDied","Data":"2b768f833e7a6bc589b28a5655a2b09598956d72c496f9471773565cca772cbf"}
May 28 17:43:48.786438 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:48.786350    2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 28 17:43:48.886183 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:48.886145    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:43:49.517343 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:49.517312    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:49.517521 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:49.517426    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:49.517521 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:49.517450    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:49.517521 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:49.517471    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:49.517696 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:49.517538    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgbhs" podUID="a8061064-d0c1-4eea-bb4c-2c237c83b89c"
May 28 17:43:49.517696 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:49.517628    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:50.246472 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:50.246439    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret\") pod \"global-pull-secret-syncer-hgbhs\" (UID: \"a8061064-d0c1-4eea-bb4c-2c237c83b89c\") " pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:50.246864 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:50.246588    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 17:43:50.246864 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:50.246661    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret podName:a8061064-d0c1-4eea-bb4c-2c237c83b89c nodeName:}" failed. No retries permitted until 2026-05-28 17:44:06.246646153 +0000 UTC m=+47.350341862 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret") pod "global-pull-secret-syncer-hgbhs" (UID: "a8061064-d0c1-4eea-bb4c-2c237c83b89c") : object "kube-system"/"original-pull-secret" not registered
May 28 17:43:50.638113 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:50.638071    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-96mcv"
May 28 17:43:50.638298 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:50.638215    2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 28 17:43:50.638735 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:50.638699    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-96mcv"
May 28 17:43:50.791769 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:50.791738    2560 generic.go:358] "Generic (PLEG): container finished" podID="f454ed50-48e4-4202-88d3-1402e75de767" containerID="95aefe6712987d95bcfd7c0c51a5b4b7502384c709b5a62a7b22e9d915c661d2" exitCode=0
May 28 17:43:50.791970 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:50.791802    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k47r7" event={"ID":"f454ed50-48e4-4202-88d3-1402e75de767","Type":"ContainerDied","Data":"95aefe6712987d95bcfd7c0c51a5b4b7502384c709b5a62a7b22e9d915c661d2"}
May 28 17:43:51.513643 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:51.513610    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:51.514175 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:51.513653    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:51.514175 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:51.513610    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:51.514175 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:51.513741    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:51.514175 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:51.513836    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:51.514175 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:51.513891    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgbhs" podUID="a8061064-d0c1-4eea-bb4c-2c237c83b89c"
May 28 17:43:52.263426 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:52.260971    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs\") pod \"network-metrics-daemon-rh246\" (UID: \"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4\") " pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:52.263426 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:52.262390    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:43:52.263426 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:52.262483    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs podName:4693da93-c00a-4dd7-8dd0-fb3be0c73ad4 nodeName:}" failed. No retries permitted until 2026-05-28 17:44:24.262461189 +0000 UTC m=+65.366156909 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs") pod "network-metrics-daemon-rh246" (UID: "4693da93-c00a-4dd7-8dd0-fb3be0c73ad4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:43:52.361505 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:52.361460    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x62nf\" (UniqueName: \"kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf\") pod \"network-check-target-ch7tt\" (UID: \"0eec8ff8-0b7b-4d4d-a787-463843df5ef7\") " pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:52.361699 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:52.361638    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:43:52.361699 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:52.361660    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:43:52.361699 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:52.361672    2560 projected.go:194] Error preparing data for projected volume kube-api-access-x62nf for pod openshift-network-diagnostics/network-check-target-ch7tt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:43:52.361857 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:52.361727    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf podName:0eec8ff8-0b7b-4d4d-a787-463843df5ef7 nodeName:}" failed. No retries permitted until 2026-05-28 17:44:24.361711681 +0000 UTC m=+65.465407391 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-x62nf" (UniqueName: "kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf") pod "network-check-target-ch7tt" (UID: "0eec8ff8-0b7b-4d4d-a787-463843df5ef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:43:53.513686 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:53.513587    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:53.514292 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:53.513598    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:53.514292 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:53.513726    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:43:53.514292 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:53.513809    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ch7tt" podUID="0eec8ff8-0b7b-4d4d-a787-463843df5ef7"
May 28 17:43:53.514292 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:53.513858    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:53.514292 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:53.513924    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgbhs" podUID="a8061064-d0c1-4eea-bb4c-2c237c83b89c"
May 28 17:43:54.733870 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.733800    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-41.ec2.internal" event="NodeReady"
May 28 17:43:54.734346 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.733961    2560 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 28 17:43:54.769141 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.769108    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-ffdf9ff96-wh6rt"]
May 28 17:43:54.774093 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.774072    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"]
May 28 17:43:54.774254 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.774231    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.776777 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.776758    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 28 17:43:54.776882 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.776775    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 28 17:43:54.776882 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.776758    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 28 17:43:54.776882 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.776834    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mvn4l\""
May 28 17:43:54.777581 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.777528    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:43:54.779314 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.779293    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cf92m"]
May 28 17:43:54.781487 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.780890    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
May 28 17:43:54.781487 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.780899    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9mcx8\""
May 28 17:43:54.781487 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.780982    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
May 28 17:43:54.782474 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.782456    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cf92m"
May 28 17:43:54.782688 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.782599    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"]
May 28 17:43:54.784956 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.783041    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 28 17:43:54.786514 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.786491    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v9cdg\""
May 28 17:43:54.787341 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.786725    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 28 17:43:54.787341 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.787000    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 28 17:43:54.787477 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.787357    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-ffdf9ff96-wh6rt"]
May 28 17:43:54.789956 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.789914    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-t9dpl"]
May 28 17:43:54.793474 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.793459    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cf92m"]
May 28 17:43:54.793575 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.793553    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:43:54.795943 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.795909    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 28 17:43:54.796064 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.796046    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8dtch\""
May 28 17:43:54.796132 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.796070    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 28 17:43:54.796186 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.796136    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 28 17:43:54.802133 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.802111    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t9dpl"]
May 28 17:43:54.879894 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.879853    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-image-registry-private-configuration\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.880061 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.879899    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kgww\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-kube-api-access-4kgww\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.880061 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.879952    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-config-volume\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:43:54.880061 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.879982    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert\") pod \"ingress-canary-t9dpl\" (UID: \"f1717a8a-a4ee-4b09-b3c2-b06dbf730baf\") " pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:43:54.880061 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.880026    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/655375f3-42df-4046-b86a-386dfee4ce6d-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-27wps\" (UID: \"655375f3-42df-4046-b86a-386dfee4ce6d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:43:54.880267 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.880111    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-certificates\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.880267 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.880139    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-trusted-ca\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.880267 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.880181    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhh4c\" (UniqueName: \"kubernetes.io/projected/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-kube-api-access-nhh4c\") pod \"ingress-canary-t9dpl\" (UID: \"f1717a8a-a4ee-4b09-b3c2-b06dbf730baf\") " pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:43:54.880267 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.880206    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.880267 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.880231    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-installation-pull-secrets\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.880267 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.880257    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-bound-sa-token\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.880538 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.880282    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs4hs\" (UniqueName: \"kubernetes.io/projected/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-kube-api-access-gs4hs\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:43:54.880538 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.880309    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:43:54.880538 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.880333    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-tmp-dir\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:43:54.880538 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.880357    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-27wps\" (UID: \"655375f3-42df-4046-b86a-386dfee4ce6d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:43:54.880538 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.880413    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-ca-trust-extracted\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.981275 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.981203    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert\") pod \"ingress-canary-t9dpl\" (UID: \"f1717a8a-a4ee-4b09-b3c2-b06dbf730baf\") " pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:43:54.981453 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.981313    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/655375f3-42df-4046-b86a-386dfee4ce6d-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-27wps\" (UID: \"655375f3-42df-4046-b86a-386dfee4ce6d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:43:54.981453 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:54.981355    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:43:54.981453 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.981376    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-certificates\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.981453 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.981405    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-trusted-ca\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.981453 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:54.981425    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert podName:f1717a8a-a4ee-4b09-b3c2-b06dbf730baf nodeName:}" failed. No retries permitted until 2026-05-28 17:43:55.481404743 +0000 UTC m=+36.585100462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert") pod "ingress-canary-t9dpl" (UID: "f1717a8a-a4ee-4b09-b3c2-b06dbf730baf") : secret "canary-serving-cert" not found
May 28 17:43:54.981727 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.981476    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nhh4c\" (UniqueName: \"kubernetes.io/projected/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-kube-api-access-nhh4c\") pod \"ingress-canary-t9dpl\" (UID: \"f1717a8a-a4ee-4b09-b3c2-b06dbf730baf\") " pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:43:54.981727 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.981505    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.981727 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.981529    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-installation-pull-secrets\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.981727 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.981557    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-bound-sa-token\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.981727 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.981583    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gs4hs\" (UniqueName: \"kubernetes.io/projected/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-kube-api-access-gs4hs\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:43:54.981727 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.981613    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:43:54.981727 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.981636    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-tmp-dir\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:43:54.981727 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.981661    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-27wps\" (UID: \"655375f3-42df-4046-b86a-386dfee4ce6d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:43:54.981727 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.981695    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-ca-trust-extracted\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.982073 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.981727    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-image-registry-private-configuration\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.982073 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.981755    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4kgww\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-kube-api-access-4kgww\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.982073 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.981794    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-config-volume\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:43:54.982187 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.982117    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/655375f3-42df-4046-b86a-386dfee4ce6d-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-27wps\" (UID: \"655375f3-42df-4046-b86a-386dfee4ce6d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:43:54.982407 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.982387    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-config-volume\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:43:54.982446 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.982412    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-certificates\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.982446 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.982427    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-trusted-ca\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.982509 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:54.982491    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:43:54.982542 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:54.982508    2560 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:43:54.982542 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:54.982519    2560 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ffdf9ff96-wh6rt: secret "image-registry-tls" not found
May 28 17:43:54.982542 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:54.982538    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls podName:b8a6a83f-2e25-446a-8bac-f1c1e9f4defc nodeName:}" failed. No retries permitted until 2026-05-28 17:43:55.482521944 +0000 UTC m=+36.586217654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls") pod "dns-default-cf92m" (UID: "b8a6a83f-2e25-446a-8bac-f1c1e9f4defc") : secret "dns-default-metrics-tls" not found
May 28 17:43:54.982630 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:54.982558    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls podName:909a7f25-5e01-448d-a17b-2eb5af1e5e5e nodeName:}" failed. No retries permitted until 2026-05-28 17:43:55.482545596 +0000 UTC m=+36.586241291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls") pod "image-registry-ffdf9ff96-wh6rt" (UID: "909a7f25-5e01-448d-a17b-2eb5af1e5e5e") : secret "image-registry-tls" not found
May 28 17:43:54.982759 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.982740    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-tmp-dir\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:43:54.982858 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:54.982797    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:43:54.982858 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:54.982847    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert podName:655375f3-42df-4046-b86a-386dfee4ce6d nodeName:}" failed. No retries permitted until 2026-05-28 17:43:55.48283086 +0000 UTC m=+36.586526557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-27wps" (UID: "655375f3-42df-4046-b86a-386dfee4ce6d") : secret "networking-console-plugin-cert" not found
May 28 17:43:54.983059 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.983034    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-ca-trust-extracted\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.987138 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.987086    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-image-registry-private-configuration\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.987245 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.987145    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-installation-pull-secrets\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.990923 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.990895    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs4hs\" (UniqueName: \"kubernetes.io/projected/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-kube-api-access-gs4hs\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:43:54.991037 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.991012    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-bound-sa-token\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:54.991575 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.991552    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhh4c\" (UniqueName: \"kubernetes.io/projected/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-kube-api-access-nhh4c\") pod \"ingress-canary-t9dpl\" (UID: \"f1717a8a-a4ee-4b09-b3c2-b06dbf730baf\") " pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:43:54.991898 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:54.991879    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kgww\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-kube-api-access-4kgww\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:55.486328 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:55.486106    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:55.486522 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:55.486347    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:43:55.486522 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:55.486374    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-27wps\" (UID: \"655375f3-42df-4046-b86a-386dfee4ce6d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:43:55.486522 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:55.486279    2560 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:43:55.486522 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:55.486421    2560 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ffdf9ff96-wh6rt: secret "image-registry-tls" not found
May 28 17:43:55.486522 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:55.486429    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert\") pod \"ingress-canary-t9dpl\" (UID: \"f1717a8a-a4ee-4b09-b3c2-b06dbf730baf\") " pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:43:55.486522 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:55.486485    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls podName:909a7f25-5e01-448d-a17b-2eb5af1e5e5e nodeName:}" failed. No retries permitted until 2026-05-28 17:43:56.486460685 +0000 UTC m=+37.590156394 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls") pod "image-registry-ffdf9ff96-wh6rt" (UID: "909a7f25-5e01-448d-a17b-2eb5af1e5e5e") : secret "image-registry-tls" not found
May 28 17:43:55.486522 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:55.486514    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:43:55.486829 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:55.486566    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:43:55.486829 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:55.486580    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls podName:b8a6a83f-2e25-446a-8bac-f1c1e9f4defc nodeName:}" failed. No retries permitted until 2026-05-28 17:43:56.486563873 +0000 UTC m=+37.590259587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls") pod "dns-default-cf92m" (UID: "b8a6a83f-2e25-446a-8bac-f1c1e9f4defc") : secret "dns-default-metrics-tls" not found
May 28 17:43:55.486829 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:55.486514    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:43:55.486829 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:55.486601    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert podName:655375f3-42df-4046-b86a-386dfee4ce6d nodeName:}" failed. No retries permitted until 2026-05-28 17:43:56.48658986 +0000 UTC m=+37.590285763 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-27wps" (UID: "655375f3-42df-4046-b86a-386dfee4ce6d") : secret "networking-console-plugin-cert" not found
May 28 17:43:55.486829 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:55.486616    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert podName:f1717a8a-a4ee-4b09-b3c2-b06dbf730baf nodeName:}" failed. No retries permitted until 2026-05-28 17:43:56.486607184 +0000 UTC m=+37.590302881 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert") pod "ingress-canary-t9dpl" (UID: "f1717a8a-a4ee-4b09-b3c2-b06dbf730baf") : secret "canary-serving-cert" not found
May 28 17:43:55.513425 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:55.513392    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:43:55.513604 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:55.513392    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:43:55.513744 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:55.513393    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:43:55.515764 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:55.515741    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 28 17:43:55.515861 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:55.515808    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5wt76\""
May 28 17:43:55.515861 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:55.515740    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 17:43:55.516718 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:55.516699    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 17:43:55.516718 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:55.516716    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 17:43:55.516877 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:55.516732    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7tvx\""
May 28 17:43:56.496081 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:56.496040    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:56.496545 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:56.496103    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:43:56.496545 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:56.496135    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-27wps\" (UID: \"655375f3-42df-4046-b86a-386dfee4ce6d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:43:56.496545 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:56.496183    2560 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:43:56.496545 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:56.496189    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert\") pod \"ingress-canary-t9dpl\" (UID: \"f1717a8a-a4ee-4b09-b3c2-b06dbf730baf\") " pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:43:56.496545 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:56.496199    2560 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ffdf9ff96-wh6rt: secret "image-registry-tls" not found
May 28 17:43:56.496545 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:56.496266    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls podName:909a7f25-5e01-448d-a17b-2eb5af1e5e5e nodeName:}" failed. No retries permitted until 2026-05-28 17:43:58.496247564 +0000 UTC m=+39.599943279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls") pod "image-registry-ffdf9ff96-wh6rt" (UID: "909a7f25-5e01-448d-a17b-2eb5af1e5e5e") : secret "image-registry-tls" not found
May 28 17:43:56.496545 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:56.496303    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:43:56.496545 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:56.496336    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert podName:f1717a8a-a4ee-4b09-b3c2-b06dbf730baf nodeName:}" failed. No retries permitted until 2026-05-28 17:43:58.496325805 +0000 UTC m=+39.600021501 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert") pod "ingress-canary-t9dpl" (UID: "f1717a8a-a4ee-4b09-b3c2-b06dbf730baf") : secret "canary-serving-cert" not found
May 28 17:43:56.496545 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:56.496384    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:43:56.496545 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:56.496411    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls podName:b8a6a83f-2e25-446a-8bac-f1c1e9f4defc nodeName:}" failed. No retries permitted until 2026-05-28 17:43:58.496401727 +0000 UTC m=+39.600097437 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls") pod "dns-default-cf92m" (UID: "b8a6a83f-2e25-446a-8bac-f1c1e9f4defc") : secret "dns-default-metrics-tls" not found
May 28 17:43:56.496545 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:56.496457    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:43:56.496545 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:56.496484    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert podName:655375f3-42df-4046-b86a-386dfee4ce6d nodeName:}" failed. No retries permitted until 2026-05-28 17:43:58.496475539 +0000 UTC m=+39.600171250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-27wps" (UID: "655375f3-42df-4046-b86a-386dfee4ce6d") : secret "networking-console-plugin-cert" not found
May 28 17:43:56.809412 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:56.809374    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k47r7" event={"ID":"f454ed50-48e4-4202-88d3-1402e75de767","Type":"ContainerStarted","Data":"1c8b6dd7391a5e808176e1d1852046cf7826507fb48a4bc53de8e9a64f908939"}
May 28 17:43:57.813412 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:57.813374    2560 generic.go:358] "Generic (PLEG): container finished" podID="f454ed50-48e4-4202-88d3-1402e75de767" containerID="1c8b6dd7391a5e808176e1d1852046cf7826507fb48a4bc53de8e9a64f908939" exitCode=0
May 28 17:43:57.813965 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:57.813426    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k47r7" event={"ID":"f454ed50-48e4-4202-88d3-1402e75de767","Type":"ContainerDied","Data":"1c8b6dd7391a5e808176e1d1852046cf7826507fb48a4bc53de8e9a64f908939"}
May 28 17:43:58.512612 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:58.512576    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert\") pod \"ingress-canary-t9dpl\" (UID: \"f1717a8a-a4ee-4b09-b3c2-b06dbf730baf\") " pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:43:58.512814 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:58.512670    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:43:58.512814 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:58.512704    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:43:58.512814 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:58.512729    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-27wps\" (UID: \"655375f3-42df-4046-b86a-386dfee4ce6d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:43:58.512814 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:58.512743    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:43:58.512814 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:58.512805    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert podName:f1717a8a-a4ee-4b09-b3c2-b06dbf730baf nodeName:}" failed. No retries permitted until 2026-05-28 17:44:02.51278526 +0000 UTC m=+43.616480977 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert") pod "ingress-canary-t9dpl" (UID: "f1717a8a-a4ee-4b09-b3c2-b06dbf730baf") : secret "canary-serving-cert" not found
May 28 17:43:58.513108 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:58.512817    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:43:58.513108 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:58.512840    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:43:58.513108 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:58.512819    2560 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:43:58.513108 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:58.512869    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert podName:655375f3-42df-4046-b86a-386dfee4ce6d nodeName:}" failed. No retries permitted until 2026-05-28 17:44:02.512861451 +0000 UTC m=+43.616557147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-27wps" (UID: "655375f3-42df-4046-b86a-386dfee4ce6d") : secret "networking-console-plugin-cert" not found
May 28 17:43:58.513108 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:58.512881    2560 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ffdf9ff96-wh6rt: secret "image-registry-tls" not found
May 28 17:43:58.513108 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:58.512886    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls podName:b8a6a83f-2e25-446a-8bac-f1c1e9f4defc nodeName:}" failed. No retries permitted until 2026-05-28 17:44:02.512880687 +0000 UTC m=+43.616576382 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls") pod "dns-default-cf92m" (UID: "b8a6a83f-2e25-446a-8bac-f1c1e9f4defc") : secret "dns-default-metrics-tls" not found
May 28 17:43:58.513108 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:43:58.512916    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls podName:909a7f25-5e01-448d-a17b-2eb5af1e5e5e nodeName:}" failed. No retries permitted until 2026-05-28 17:44:02.512897335 +0000 UTC m=+43.616593031 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls") pod "image-registry-ffdf9ff96-wh6rt" (UID: "909a7f25-5e01-448d-a17b-2eb5af1e5e5e") : secret "image-registry-tls" not found
May 28 17:43:58.818272 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:58.818236    2560 generic.go:358] "Generic (PLEG): container finished" podID="f454ed50-48e4-4202-88d3-1402e75de767" containerID="13cc9bcb81b32961b8782f22b424db2488d762a08968aff20aa96d4a9ec4b1bd" exitCode=0
May 28 17:43:58.818698 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:58.818312    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k47r7" event={"ID":"f454ed50-48e4-4202-88d3-1402e75de767","Type":"ContainerDied","Data":"13cc9bcb81b32961b8782f22b424db2488d762a08968aff20aa96d4a9ec4b1bd"}
May 28 17:43:59.823162 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:59.823128    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k47r7" event={"ID":"f454ed50-48e4-4202-88d3-1402e75de767","Type":"ContainerStarted","Data":"5244dfa1f16613e9a1ed20a27d6a797cd8f83732b83f4d3d063ad6fd3cc1127b"}
May 28 17:43:59.845243 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:43:59.845190    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-k47r7" podStartSLOduration=5.106572484 podStartE2EDuration="40.845175484s" podCreationTimestamp="2026-05-28 17:43:19 +0000 UTC" firstStartedPulling="2026-05-28 17:43:20.869733292 +0000 UTC m=+1.973429003" lastFinishedPulling="2026-05-28 17:43:56.608336303 +0000 UTC m=+37.712032003" observedRunningTime="2026-05-28 17:43:59.844649517 +0000 UTC m=+40.948345235" watchObservedRunningTime="2026-05-28 17:43:59.845175484 +0000 UTC m=+40.948871202"
May 28 17:44:02.539613 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:02.539579    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert\") pod \"ingress-canary-t9dpl\" (UID: \"f1717a8a-a4ee-4b09-b3c2-b06dbf730baf\") " pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:44:02.540025 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:02.539669    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:44:02.540025 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:02.539691    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:44:02.540025 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:02.539710    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-27wps\" (UID: \"655375f3-42df-4046-b86a-386dfee4ce6d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:44:02.540025 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:02.539758    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:44:02.540025 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:02.539800    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:44:02.540025 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:02.539808    2560 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:44:02.540025 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:02.539820    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:44:02.540025 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:02.539845    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert podName:f1717a8a-a4ee-4b09-b3c2-b06dbf730baf nodeName:}" failed. No retries permitted until 2026-05-28 17:44:10.539823948 +0000 UTC m=+51.643519660 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert") pod "ingress-canary-t9dpl" (UID: "f1717a8a-a4ee-4b09-b3c2-b06dbf730baf") : secret "canary-serving-cert" not found
May 28 17:44:02.540025 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:02.539867    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls podName:b8a6a83f-2e25-446a-8bac-f1c1e9f4defc nodeName:}" failed. No retries permitted until 2026-05-28 17:44:10.539854828 +0000 UTC m=+51.643550527 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls") pod "dns-default-cf92m" (UID: "b8a6a83f-2e25-446a-8bac-f1c1e9f4defc") : secret "dns-default-metrics-tls" not found
May 28 17:44:02.540025 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:02.539881    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert podName:655375f3-42df-4046-b86a-386dfee4ce6d nodeName:}" failed. No retries permitted until 2026-05-28 17:44:10.539874206 +0000 UTC m=+51.643569903 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-27wps" (UID: "655375f3-42df-4046-b86a-386dfee4ce6d") : secret "networking-console-plugin-cert" not found
May 28 17:44:02.540025 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:02.539825    2560 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ffdf9ff96-wh6rt: secret "image-registry-tls" not found
May 28 17:44:02.540025 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:02.539905    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls podName:909a7f25-5e01-448d-a17b-2eb5af1e5e5e nodeName:}" failed. No retries permitted until 2026-05-28 17:44:10.539900015 +0000 UTC m=+51.643595711 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls") pod "image-registry-ffdf9ff96-wh6rt" (UID: "909a7f25-5e01-448d-a17b-2eb5af1e5e5e") : secret "image-registry-tls" not found
May 28 17:44:06.077796 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.077761    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9"]
May 28 17:44:06.118401 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.118372    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9"]
May 28 17:44:06.118401 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.118398    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"]
May 28 17:44:06.118594 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.118511    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9"
May 28 17:44:06.120943 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.120897    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-wxv7x\""
May 28 17:44:06.120943 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.120898    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
May 28 17:44:06.121853 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.121803    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
May 28 17:44:06.121853 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.121837    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
May 28 17:44:06.122107 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.121876    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
May 28 17:44:06.134326 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.134302    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"]
May 28 17:44:06.134439 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.134389    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.136105 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.136083    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"]
May 28 17:44:06.136728 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.136712    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
May 28 17:44:06.136797 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.136714    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
May 28 17:44:06.136797 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.136764    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
May 28 17:44:06.137128 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.137115    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
May 28 17:44:06.152496 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.152471    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"]
May 28 17:44:06.152645 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.152600    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"
May 28 17:44:06.155214 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.155194    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
May 28 17:44:06.269048 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.269009    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff40f739-b416-49f9-8397-9e1c54a743fc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9\" (UID: \"ff40f739-b416-49f9-8397-9e1c54a743fc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9"
May 28 17:44:06.269048 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.269048    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/9c219cc6-46bf-4924-b97b-3285fb0d799b-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.269300 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.269082    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqqc2\" (UniqueName: \"kubernetes.io/projected/2b3a6a5a-c520-494b-8399-d47da5508054-kube-api-access-vqqc2\") pod \"klusterlet-addon-workmgr-5b6664dc8f-zxsjn\" (UID: \"2b3a6a5a-c520-494b-8399-d47da5508054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"
May 28 17:44:06.269300 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.269120    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b3a6a5a-c520-494b-8399-d47da5508054-tmp\") pod \"klusterlet-addon-workmgr-5b6664dc8f-zxsjn\" (UID: \"2b3a6a5a-c520-494b-8399-d47da5508054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"
May 28 17:44:06.269300 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.269156    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9c219cc6-46bf-4924-b97b-3285fb0d799b-ca\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.269300 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.269177    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h74lp\" (UniqueName: \"kubernetes.io/projected/ff40f739-b416-49f9-8397-9e1c54a743fc-kube-api-access-h74lp\") pod \"managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9\" (UID: \"ff40f739-b416-49f9-8397-9e1c54a743fc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9"
May 28 17:44:06.269300 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.269259    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9c219cc6-46bf-4924-b97b-3285fb0d799b-hub\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.269496 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.269303    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgwxl\" (UniqueName: \"kubernetes.io/projected/9c219cc6-46bf-4924-b97b-3285fb0d799b-kube-api-access-rgwxl\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.269496 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.269337    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9c219cc6-46bf-4924-b97b-3285fb0d799b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.269496 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.269385    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9c219cc6-46bf-4924-b97b-3285fb0d799b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.269496 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.269440    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret\") pod \"global-pull-secret-syncer-hgbhs\" (UID: \"a8061064-d0c1-4eea-bb4c-2c237c83b89c\") " pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:44:06.269496 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.269465    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2b3a6a5a-c520-494b-8399-d47da5508054-klusterlet-config\") pod \"klusterlet-addon-workmgr-5b6664dc8f-zxsjn\" (UID: \"2b3a6a5a-c520-494b-8399-d47da5508054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"
May 28 17:44:06.272678 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.272418    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a8061064-d0c1-4eea-bb4c-2c237c83b89c-original-pull-secret\") pod \"global-pull-secret-syncer-hgbhs\" (UID: \"a8061064-d0c1-4eea-bb4c-2c237c83b89c\") " pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:44:06.332629 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.332553    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgbhs"
May 28 17:44:06.370910 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.370869    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff40f739-b416-49f9-8397-9e1c54a743fc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9\" (UID: \"ff40f739-b416-49f9-8397-9e1c54a743fc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9"
May 28 17:44:06.371124 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.370922    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/9c219cc6-46bf-4924-b97b-3285fb0d799b-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.371124 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.370978    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vqqc2\" (UniqueName: \"kubernetes.io/projected/2b3a6a5a-c520-494b-8399-d47da5508054-kube-api-access-vqqc2\") pod \"klusterlet-addon-workmgr-5b6664dc8f-zxsjn\" (UID: \"2b3a6a5a-c520-494b-8399-d47da5508054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"
May 28 17:44:06.371124 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.371015    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b3a6a5a-c520-494b-8399-d47da5508054-tmp\") pod \"klusterlet-addon-workmgr-5b6664dc8f-zxsjn\" (UID: \"2b3a6a5a-c520-494b-8399-d47da5508054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"
May 28 17:44:06.371124 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.371068    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9c219cc6-46bf-4924-b97b-3285fb0d799b-ca\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.371124 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.371101    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h74lp\" (UniqueName: \"kubernetes.io/projected/ff40f739-b416-49f9-8397-9e1c54a743fc-kube-api-access-h74lp\") pod \"managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9\" (UID: \"ff40f739-b416-49f9-8397-9e1c54a743fc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9"
May 28 17:44:06.371371 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.371145    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9c219cc6-46bf-4924-b97b-3285fb0d799b-hub\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.371371 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.371172    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rgwxl\" (UniqueName: \"kubernetes.io/projected/9c219cc6-46bf-4924-b97b-3285fb0d799b-kube-api-access-rgwxl\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.371371 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.371200    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9c219cc6-46bf-4924-b97b-3285fb0d799b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.371371 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.371238    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9c219cc6-46bf-4924-b97b-3285fb0d799b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.371371 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.371286    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2b3a6a5a-c520-494b-8399-d47da5508054-klusterlet-config\") pod \"klusterlet-addon-workmgr-5b6664dc8f-zxsjn\" (UID: \"2b3a6a5a-c520-494b-8399-d47da5508054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"
May 28 17:44:06.372138 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.372073    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b3a6a5a-c520-494b-8399-d47da5508054-tmp\") pod \"klusterlet-addon-workmgr-5b6664dc8f-zxsjn\" (UID: \"2b3a6a5a-c520-494b-8399-d47da5508054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"
May 28 17:44:06.372416 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.372386    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/9c219cc6-46bf-4924-b97b-3285fb0d799b-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.374309 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.374279    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9c219cc6-46bf-4924-b97b-3285fb0d799b-ca\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.374943 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.374903    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2b3a6a5a-c520-494b-8399-d47da5508054-klusterlet-config\") pod \"klusterlet-addon-workmgr-5b6664dc8f-zxsjn\" (UID: \"2b3a6a5a-c520-494b-8399-d47da5508054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"
May 28 17:44:06.375067 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.375050    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9c219cc6-46bf-4924-b97b-3285fb0d799b-hub\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.375147 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.375124    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9c219cc6-46bf-4924-b97b-3285fb0d799b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.375209 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.375188    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff40f739-b416-49f9-8397-9e1c54a743fc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9\" (UID: \"ff40f739-b416-49f9-8397-9e1c54a743fc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9"
May 28 17:44:06.375263 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.375209    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9c219cc6-46bf-4924-b97b-3285fb0d799b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.380055 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.380027    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h74lp\" (UniqueName: \"kubernetes.io/projected/ff40f739-b416-49f9-8397-9e1c54a743fc-kube-api-access-h74lp\") pod \"managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9\" (UID: \"ff40f739-b416-49f9-8397-9e1c54a743fc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9"
May 28 17:44:06.380252 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.380235    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgwxl\" (UniqueName: \"kubernetes.io/projected/9c219cc6-46bf-4924-b97b-3285fb0d799b-kube-api-access-rgwxl\") pod \"cluster-proxy-proxy-agent-7cf5d84b67-zd86d\" (UID: \"9c219cc6-46bf-4924-b97b-3285fb0d799b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.380320 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.380235    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqqc2\" (UniqueName: \"kubernetes.io/projected/2b3a6a5a-c520-494b-8399-d47da5508054-kube-api-access-vqqc2\") pod \"klusterlet-addon-workmgr-5b6664dc8f-zxsjn\" (UID: \"2b3a6a5a-c520-494b-8399-d47da5508054\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"
May 28 17:44:06.439092 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.439060    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9"
May 28 17:44:06.445486 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.445466    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:44:06.471686 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.471645    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"
May 28 17:44:06.549254 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.549205    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hgbhs"]
May 28 17:44:06.607900 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.607872    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"]
May 28 17:44:06.611810 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:44:06.611768    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c219cc6_46bf_4924_b97b_3285fb0d799b.slice/crio-b77e3486e168d9789c7ad4d6a1be1aba5ec7243e29a337deb9ca5d9ada596f87 WatchSource:0}: Error finding container b77e3486e168d9789c7ad4d6a1be1aba5ec7243e29a337deb9ca5d9ada596f87: Status 404 returned error can't find the container with id b77e3486e168d9789c7ad4d6a1be1aba5ec7243e29a337deb9ca5d9ada596f87
May 28 17:44:06.621800 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.621768    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9"]
May 28 17:44:06.624447 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:44:06.624422    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff40f739_b416_49f9_8397_9e1c54a743fc.slice/crio-4fef80448521a5a673d5573f7030032a8d7bd795ed79211e5c2da91f1f025591 WatchSource:0}: Error finding container 4fef80448521a5a673d5573f7030032a8d7bd795ed79211e5c2da91f1f025591: Status 404 returned error can't find the container with id 4fef80448521a5a673d5573f7030032a8d7bd795ed79211e5c2da91f1f025591
May 28 17:44:06.645709 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.645678    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"]
May 28 17:44:06.648099 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:44:06.648067    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b3a6a5a_c520_494b_8399_d47da5508054.slice/crio-870b046ef219fe47187293c50bd395edc817453a82058f9c93fdd7e4e516c79d WatchSource:0}: Error finding container 870b046ef219fe47187293c50bd395edc817453a82058f9c93fdd7e4e516c79d: Status 404 returned error can't find the container with id 870b046ef219fe47187293c50bd395edc817453a82058f9c93fdd7e4e516c79d
May 28 17:44:06.837694 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.837655    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hgbhs" event={"ID":"a8061064-d0c1-4eea-bb4c-2c237c83b89c","Type":"ContainerStarted","Data":"10bc01b7ea2a1a2f5b02e5bb3a79e15f675fb6d7bf9e2bd831cfc4536b9f5ff5"}
May 28 17:44:06.838682 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.838650    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9" event={"ID":"ff40f739-b416-49f9-8397-9e1c54a743fc","Type":"ContainerStarted","Data":"4fef80448521a5a673d5573f7030032a8d7bd795ed79211e5c2da91f1f025591"}
May 28 17:44:06.839671 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.839648    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d" event={"ID":"9c219cc6-46bf-4924-b97b-3285fb0d799b","Type":"ContainerStarted","Data":"b77e3486e168d9789c7ad4d6a1be1aba5ec7243e29a337deb9ca5d9ada596f87"}
May 28 17:44:06.840551 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:06.840531    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn" event={"ID":"2b3a6a5a-c520-494b-8399-d47da5508054","Type":"ContainerStarted","Data":"870b046ef219fe47187293c50bd395edc817453a82058f9c93fdd7e4e516c79d"}
May 28 17:44:10.607626 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:10.607586    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert\") pod \"ingress-canary-t9dpl\" (UID: \"f1717a8a-a4ee-4b09-b3c2-b06dbf730baf\") " pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:44:10.608150 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:10.607651    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:44:10.608150 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:10.607697    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:44:10.608150 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:10.607732    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-27wps\" (UID: \"655375f3-42df-4046-b86a-386dfee4ce6d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:44:10.608150 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:10.607742    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:44:10.608150 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:10.607766    2560 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:44:10.608150 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:10.607787    2560 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ffdf9ff96-wh6rt: secret "image-registry-tls" not found
May 28 17:44:10.608150 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:10.607814    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert podName:f1717a8a-a4ee-4b09-b3c2-b06dbf730baf nodeName:}" failed. No retries permitted until 2026-05-28 17:44:26.607799207 +0000 UTC m=+67.711494906 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert") pod "ingress-canary-t9dpl" (UID: "f1717a8a-a4ee-4b09-b3c2-b06dbf730baf") : secret "canary-serving-cert" not found
May 28 17:44:10.608150 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:10.607820    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:44:10.608150 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:10.607844    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls podName:909a7f25-5e01-448d-a17b-2eb5af1e5e5e nodeName:}" failed. No retries permitted until 2026-05-28 17:44:26.607824518 +0000 UTC m=+67.711520229 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls") pod "image-registry-ffdf9ff96-wh6rt" (UID: "909a7f25-5e01-448d-a17b-2eb5af1e5e5e") : secret "image-registry-tls" not found
May 28 17:44:10.608150 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:10.607855    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:44:10.608150 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:10.607863    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert podName:655375f3-42df-4046-b86a-386dfee4ce6d nodeName:}" failed. No retries permitted until 2026-05-28 17:44:26.607852968 +0000 UTC m=+67.711548668 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-27wps" (UID: "655375f3-42df-4046-b86a-386dfee4ce6d") : secret "networking-console-plugin-cert" not found
May 28 17:44:10.608150 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:10.607950    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls podName:b8a6a83f-2e25-446a-8bac-f1c1e9f4defc nodeName:}" failed. No retries permitted until 2026-05-28 17:44:26.607915251 +0000 UTC m=+67.711610947 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls") pod "dns-default-cf92m" (UID: "b8a6a83f-2e25-446a-8bac-f1c1e9f4defc") : secret "dns-default-metrics-tls" not found
May 28 17:44:15.863802 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:15.863758    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9" event={"ID":"ff40f739-b416-49f9-8397-9e1c54a743fc","Type":"ContainerStarted","Data":"07beb20c5088dd7a4a34de28afc971e01b31f9ceea1115a0630d37751fe23fb5"}
May 28 17:44:15.864922 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:15.864898    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d" event={"ID":"9c219cc6-46bf-4924-b97b-3285fb0d799b","Type":"ContainerStarted","Data":"e7beb52f70291c2438bacd0e3bcf3cc34739918fc110fd2ad2c7add7071deb08"}
May 28 17:44:15.865994 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:15.865974    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn" event={"ID":"2b3a6a5a-c520-494b-8399-d47da5508054","Type":"ContainerStarted","Data":"1cb7584e3e1bba75ea4e8e6128f5b3eced0464a1581cec38316b1387f208ef8d"}
May 28 17:44:15.866204 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:15.866187    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"
May 28 17:44:15.867457 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:15.867435    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hgbhs" event={"ID":"a8061064-d0c1-4eea-bb4c-2c237c83b89c","Type":"ContainerStarted","Data":"3657c45841e072d2ef632c6a5fabe41e333d773bbbd79cdb371c2fb079faab24"}
May 28 17:44:15.868112 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:15.868094    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"
May 28 17:44:15.878190 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:15.878143    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9" podStartSLOduration=1.190576625 podStartE2EDuration="9.878128307s" podCreationTimestamp="2026-05-28 17:44:06 +0000 UTC" firstStartedPulling="2026-05-28 17:44:06.62646623 +0000 UTC m=+47.730161942" lastFinishedPulling="2026-05-28 17:44:15.314017928 +0000 UTC m=+56.417713624" observedRunningTime="2026-05-28 17:44:15.877696477 +0000 UTC m=+56.981392193" watchObservedRunningTime="2026-05-28 17:44:15.878128307 +0000 UTC m=+56.981824027"
May 28 17:44:15.894086 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:15.894029    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn" podStartSLOduration=1.242562219 podStartE2EDuration="9.89400925s" podCreationTimestamp="2026-05-28 17:44:06 +0000 UTC" firstStartedPulling="2026-05-28 17:44:06.649863619 +0000 UTC m=+47.753559319" lastFinishedPulling="2026-05-28 17:44:15.301310651 +0000 UTC m=+56.405006350" observedRunningTime="2026-05-28 17:44:15.893428914 +0000 UTC m=+56.997124632" watchObservedRunningTime="2026-05-28 17:44:15.89400925 +0000 UTC m=+56.997704972"
May 28 17:44:15.907487 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:15.907447    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hgbhs" podStartSLOduration=33.216753433 podStartE2EDuration="41.907438906s" podCreationTimestamp="2026-05-28 17:43:34 +0000 UTC" firstStartedPulling="2026-05-28 17:44:06.559908187 +0000 UTC m=+47.663603889" lastFinishedPulling="2026-05-28 17:44:15.250593659 +0000 UTC m=+56.354289362" observedRunningTime="2026-05-28 17:44:15.906478484 +0000 UTC m=+57.010174201" watchObservedRunningTime="2026-05-28 17:44:15.907438906 +0000 UTC m=+57.011134624"
May 28 17:44:17.874578 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:17.874543    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d" event={"ID":"9c219cc6-46bf-4924-b97b-3285fb0d799b","Type":"ContainerStarted","Data":"83dcde622c6fc59cb31b5aac017841a9650944fcb7f35770fc193537425eeaab"}
May 28 17:44:17.874578 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:17.874580    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d" event={"ID":"9c219cc6-46bf-4924-b97b-3285fb0d799b","Type":"ContainerStarted","Data":"6efaeeb66923084f8fa001c0de479469dba0b24b25ea874df870d6d521209014"}
May 28 17:44:20.802889 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:20.802861    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fsvt7"
May 28 17:44:20.827521 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:20.827482    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d" podStartSLOduration=3.9851976159999998 podStartE2EDuration="14.827469617s" podCreationTimestamp="2026-05-28 17:44:06 +0000 UTC" firstStartedPulling="2026-05-28 17:44:06.614196484 +0000 UTC m=+47.717892180" lastFinishedPulling="2026-05-28 17:44:17.456468482 +0000 UTC m=+58.560164181" observedRunningTime="2026-05-28 17:44:17.895664763 +0000 UTC m=+58.999360482" watchObservedRunningTime="2026-05-28 17:44:20.827469617 +0000 UTC m=+61.931165335"
May 28 17:44:24.318220 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:24.318159    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs\") pod \"network-metrics-daemon-rh246\" (UID: \"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4\") " pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:44:24.320874 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:24.320852    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 17:44:24.328914 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:24.328892    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 17:44:24.329021 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:24.328983    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs podName:4693da93-c00a-4dd7-8dd0-fb3be0c73ad4 nodeName:}" failed. No retries permitted until 2026-05-28 17:45:28.328966623 +0000 UTC m=+129.432662319 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs") pod "network-metrics-daemon-rh246" (UID: "4693da93-c00a-4dd7-8dd0-fb3be0c73ad4") : secret "metrics-daemon-secret" not found
May 28 17:44:24.418963 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:24.418910    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x62nf\" (UniqueName: \"kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf\") pod \"network-check-target-ch7tt\" (UID: \"0eec8ff8-0b7b-4d4d-a787-463843df5ef7\") " pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:44:24.421632 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:24.421612    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 17:44:24.431450 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:24.431421    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 17:44:24.442653 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:24.442623    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x62nf\" (UniqueName: \"kubernetes.io/projected/0eec8ff8-0b7b-4d4d-a787-463843df5ef7-kube-api-access-x62nf\") pod \"network-check-target-ch7tt\" (UID: \"0eec8ff8-0b7b-4d4d-a787-463843df5ef7\") " pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:44:24.640652 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:24.640566    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7tvx\""
May 28 17:44:24.648398 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:24.648377    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:44:24.770843 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:24.770807    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ch7tt"]
May 28 17:44:24.773543 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:44:24.773513    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eec8ff8_0b7b_4d4d_a787_463843df5ef7.slice/crio-b2d07eb48f268addf1be790a11a36abe6d961114411fa44f1167f7eae26db969 WatchSource:0}: Error finding container b2d07eb48f268addf1be790a11a36abe6d961114411fa44f1167f7eae26db969: Status 404 returned error can't find the container with id b2d07eb48f268addf1be790a11a36abe6d961114411fa44f1167f7eae26db969
May 28 17:44:24.894118 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:24.894027    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ch7tt" event={"ID":"0eec8ff8-0b7b-4d4d-a787-463843df5ef7","Type":"ContainerStarted","Data":"b2d07eb48f268addf1be790a11a36abe6d961114411fa44f1167f7eae26db969"}
May 28 17:44:26.635691 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:26.635641    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert\") pod \"ingress-canary-t9dpl\" (UID: \"f1717a8a-a4ee-4b09-b3c2-b06dbf730baf\") " pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:44:26.636196 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:26.635720    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:44:26.636196 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:26.635767    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:44:26.636196 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:26.635809    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-27wps\" (UID: \"655375f3-42df-4046-b86a-386dfee4ce6d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:44:26.636196 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:26.635814    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:44:26.636196 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:26.635899    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert podName:f1717a8a-a4ee-4b09-b3c2-b06dbf730baf nodeName:}" failed. No retries permitted until 2026-05-28 17:44:58.635877494 +0000 UTC m=+99.739573194 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert") pod "ingress-canary-t9dpl" (UID: "f1717a8a-a4ee-4b09-b3c2-b06dbf730baf") : secret "canary-serving-cert" not found
May 28 17:44:26.636196 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:26.635900    2560 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:44:26.636196 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:26.635909    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:44:26.636196 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:26.635920    2560 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ffdf9ff96-wh6rt: secret "image-registry-tls" not found
May 28 17:44:26.636196 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:26.635976    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:44:26.636196 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:26.635980    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert podName:655375f3-42df-4046-b86a-386dfee4ce6d nodeName:}" failed. No retries permitted until 2026-05-28 17:44:58.635967892 +0000 UTC m=+99.739663602 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-27wps" (UID: "655375f3-42df-4046-b86a-386dfee4ce6d") : secret "networking-console-plugin-cert" not found
May 28 17:44:26.636196 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:26.636064    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls podName:909a7f25-5e01-448d-a17b-2eb5af1e5e5e nodeName:}" failed. No retries permitted until 2026-05-28 17:44:58.63604403 +0000 UTC m=+99.739739740 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls") pod "image-registry-ffdf9ff96-wh6rt" (UID: "909a7f25-5e01-448d-a17b-2eb5af1e5e5e") : secret "image-registry-tls" not found
May 28 17:44:26.636196 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:26.636079    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls podName:b8a6a83f-2e25-446a-8bac-f1c1e9f4defc nodeName:}" failed. No retries permitted until 2026-05-28 17:44:58.636070304 +0000 UTC m=+99.739766007 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls") pod "dns-default-cf92m" (UID: "b8a6a83f-2e25-446a-8bac-f1c1e9f4defc") : secret "dns-default-metrics-tls" not found
May 28 17:44:28.904734 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:28.904697    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ch7tt" event={"ID":"0eec8ff8-0b7b-4d4d-a787-463843df5ef7","Type":"ContainerStarted","Data":"ce1542c2b9eb5ea9db7b7d1ec6ad20c73ad4d828b5c53d4b9f48c02b4debc497"}
May 28 17:44:28.905122 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:28.904766    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:44:28.919451 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:28.919409    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ch7tt" podStartSLOduration=66.520741382 podStartE2EDuration="1m9.919397303s" podCreationTimestamp="2026-05-28 17:43:19 +0000 UTC" firstStartedPulling="2026-05-28 17:44:24.775258753 +0000 UTC m=+65.878954452" lastFinishedPulling="2026-05-28 17:44:28.173914674 +0000 UTC m=+69.277610373" observedRunningTime="2026-05-28 17:44:28.918601913 +0000 UTC m=+70.022297631" watchObservedRunningTime="2026-05-28 17:44:28.919397303 +0000 UTC m=+70.023093017"
May 28 17:44:58.689477 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:58.689435    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:44:58.689971 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:58.689490    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:44:58.689971 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:58.689520    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-27wps\" (UID: \"655375f3-42df-4046-b86a-386dfee4ce6d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:44:58.689971 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:58.689546    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert\") pod \"ingress-canary-t9dpl\" (UID: \"f1717a8a-a4ee-4b09-b3c2-b06dbf730baf\") " pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:44:58.689971 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:58.689594    2560 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:44:58.689971 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:58.689618    2560 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-ffdf9ff96-wh6rt: secret "image-registry-tls" not found
May 28 17:44:58.689971 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:58.689644    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:44:58.689971 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:58.689649    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:44:58.689971 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:58.689679    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:44:58.689971 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:58.689681    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls podName:909a7f25-5e01-448d-a17b-2eb5af1e5e5e nodeName:}" failed. No retries permitted until 2026-05-28 17:46:02.68966619 +0000 UTC m=+163.793361905 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls") pod "image-registry-ffdf9ff96-wh6rt" (UID: "909a7f25-5e01-448d-a17b-2eb5af1e5e5e") : secret "image-registry-tls" not found
May 28 17:44:58.689971 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:58.689744    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls podName:b8a6a83f-2e25-446a-8bac-f1c1e9f4defc nodeName:}" failed. No retries permitted until 2026-05-28 17:46:02.689729978 +0000 UTC m=+163.793425675 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls") pod "dns-default-cf92m" (UID: "b8a6a83f-2e25-446a-8bac-f1c1e9f4defc") : secret "dns-default-metrics-tls" not found
May 28 17:44:58.689971 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:58.689754    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert podName:f1717a8a-a4ee-4b09-b3c2-b06dbf730baf nodeName:}" failed. No retries permitted until 2026-05-28 17:46:02.689748983 +0000 UTC m=+163.793444679 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert") pod "ingress-canary-t9dpl" (UID: "f1717a8a-a4ee-4b09-b3c2-b06dbf730baf") : secret "canary-serving-cert" not found
May 28 17:44:58.689971 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:44:58.689769    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert podName:655375f3-42df-4046-b86a-386dfee4ce6d nodeName:}" failed. No retries permitted until 2026-05-28 17:46:02.689760297 +0000 UTC m=+163.793455998 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-27wps" (UID: "655375f3-42df-4046-b86a-386dfee4ce6d") : secret "networking-console-plugin-cert" not found
May 28 17:44:59.911557 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:44:59.911529    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ch7tt"
May 28 17:45:28.412459 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:45:28.412409    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs\") pod \"network-metrics-daemon-rh246\" (UID: \"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4\") " pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:45:28.412973 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:45:28.412550    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 17:45:28.412973 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:45:28.412626    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs podName:4693da93-c00a-4dd7-8dd0-fb3be0c73ad4 nodeName:}" failed. No retries permitted until 2026-05-28 17:47:30.412609917 +0000 UTC m=+251.516305639 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs") pod "network-metrics-daemon-rh246" (UID: "4693da93-c00a-4dd7-8dd0-fb3be0c73ad4") : secret "metrics-daemon-secret" not found
May 28 17:45:51.964086 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:45:51.964056    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5mv2t_70ec7ff0-1089-4ca1-8238-da94d4b1c031/dns-node-resolver/0.log"
May 28 17:45:52.963205 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:45:52.963180    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-46l5q_ec78263b-aee2-442b-8bc0-5f738c3d6832/node-ca/0.log"
May 28 17:45:57.790053 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:45:57.790013    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt" podUID="909a7f25-5e01-448d-a17b-2eb5af1e5e5e"
May 28 17:45:57.801564 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:45:57.801516    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps" podUID="655375f3-42df-4046-b86a-386dfee4ce6d"
May 28 17:45:57.808286 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:45:57.808259    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-cf92m" podUID="b8a6a83f-2e25-446a-8bac-f1c1e9f4defc"
May 28 17:45:57.814835 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:45:57.814814    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-t9dpl" podUID="f1717a8a-a4ee-4b09-b3c2-b06dbf730baf"
May 28 17:45:58.109978 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:45:58.109922    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:45:58.109978 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:45:58.109978    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cf92m"
May 28 17:45:58.110245 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:45:58.110154    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:45:58.110302 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:45:58.110264    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:45:58.527226 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:45:58.527188    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rh246" podUID="4693da93-c00a-4dd7-8dd0-fb3be0c73ad4"
May 28 17:46:02.786654 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:02.786612    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-27wps\" (UID: \"655375f3-42df-4046-b86a-386dfee4ce6d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:46:02.786654 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:02.786659    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert\") pod \"ingress-canary-t9dpl\" (UID: \"f1717a8a-a4ee-4b09-b3c2-b06dbf730baf\") " pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:46:02.787219 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:02.786694    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:46:02.787219 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:02.786721    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:46:02.787219 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:46:02.786772    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:46:02.787219 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:46:02.786857    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert podName:655375f3-42df-4046-b86a-386dfee4ce6d nodeName:}" failed. No retries permitted until 2026-05-28 17:48:04.786840555 +0000 UTC m=+285.890536254 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-27wps" (UID: "655375f3-42df-4046-b86a-386dfee4ce6d") : secret "networking-console-plugin-cert" not found
May 28 17:46:02.789250 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:02.789222    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8a6a83f-2e25-446a-8bac-f1c1e9f4defc-metrics-tls\") pod \"dns-default-cf92m\" (UID: \"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc\") " pod="openshift-dns/dns-default-cf92m"
May 28 17:46:02.789363 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:02.789340    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls\") pod \"image-registry-ffdf9ff96-wh6rt\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") " pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:46:02.789419 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:02.789369    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1717a8a-a4ee-4b09-b3c2-b06dbf730baf-cert\") pod \"ingress-canary-t9dpl\" (UID: \"f1717a8a-a4ee-4b09-b3c2-b06dbf730baf\") " pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:46:02.914203 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:02.914158    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v9cdg\""
May 28 17:46:02.914203 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:02.914177    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8dtch\""
May 28 17:46:02.914432 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:02.914209    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mvn4l\""
May 28 17:46:02.921346 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:02.921325    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:46:02.921405 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:02.921355    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cf92m"
May 28 17:46:02.921439 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:02.921405    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t9dpl"
May 28 17:46:03.071221 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:03.071167    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t9dpl"]
May 28 17:46:03.076545 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:46:03.076518    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1717a8a_a4ee_4b09_b3c2_b06dbf730baf.slice/crio-5c6b11f8f4ac2d5891fb00118b898563e90c1fb8216cb00670ac45ff04aaa221 WatchSource:0}: Error finding container 5c6b11f8f4ac2d5891fb00118b898563e90c1fb8216cb00670ac45ff04aaa221: Status 404 returned error can't find the container with id 5c6b11f8f4ac2d5891fb00118b898563e90c1fb8216cb00670ac45ff04aaa221
May 28 17:46:03.082843 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:03.082813    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cf92m"]
May 28 17:46:03.087354 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:46:03.087320    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a6a83f_2e25_446a_8bac_f1c1e9f4defc.slice/crio-88429357cfaa9b9bbbed34237171bcb932a6c083192c4d00da5ee229b2f287fe WatchSource:0}: Error finding container 88429357cfaa9b9bbbed34237171bcb932a6c083192c4d00da5ee229b2f287fe: Status 404 returned error can't find the container with id 88429357cfaa9b9bbbed34237171bcb932a6c083192c4d00da5ee229b2f287fe
May 28 17:46:03.107779 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:03.107747    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-ffdf9ff96-wh6rt"]
May 28 17:46:03.110371 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:46:03.110345    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod909a7f25_5e01_448d_a17b_2eb5af1e5e5e.slice/crio-c403e52fd461533ba99eaafa331f4635b7421d8082279d313456af21c6cb327e WatchSource:0}: Error finding container c403e52fd461533ba99eaafa331f4635b7421d8082279d313456af21c6cb327e: Status 404 returned error can't find the container with id c403e52fd461533ba99eaafa331f4635b7421d8082279d313456af21c6cb327e
May 28 17:46:03.121569 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:03.121535    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt" event={"ID":"909a7f25-5e01-448d-a17b-2eb5af1e5e5e","Type":"ContainerStarted","Data":"c403e52fd461533ba99eaafa331f4635b7421d8082279d313456af21c6cb327e"}
May 28 17:46:03.122651 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:03.122628    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cf92m" event={"ID":"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc","Type":"ContainerStarted","Data":"88429357cfaa9b9bbbed34237171bcb932a6c083192c4d00da5ee229b2f287fe"}
May 28 17:46:03.123550 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:03.123530    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t9dpl" event={"ID":"f1717a8a-a4ee-4b09-b3c2-b06dbf730baf","Type":"ContainerStarted","Data":"5c6b11f8f4ac2d5891fb00118b898563e90c1fb8216cb00670ac45ff04aaa221"}
May 28 17:46:04.128250 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:04.128199    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt" event={"ID":"909a7f25-5e01-448d-a17b-2eb5af1e5e5e","Type":"ContainerStarted","Data":"fb5c309385af9d9fa083876984723f6afe5076755581037a73bb64bbffe34eb4"}
May 28 17:46:04.128715 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:04.128447    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:46:04.148977 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:04.148908    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt" podStartSLOduration=164.148891005 podStartE2EDuration="2m44.148891005s" podCreationTimestamp="2026-05-28 17:43:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:46:04.147874109 +0000 UTC m=+165.251569828" watchObservedRunningTime="2026-05-28 17:46:04.148891005 +0000 UTC m=+165.252586727"
May 28 17:46:05.134885 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:05.134828    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t9dpl" event={"ID":"f1717a8a-a4ee-4b09-b3c2-b06dbf730baf","Type":"ContainerStarted","Data":"f8505fe57844ca3439db2fbdde77c610661b8cb21a45c5a44d853c9a5ef41c3b"}
May 28 17:46:05.137196 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:05.137151    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cf92m" event={"ID":"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc","Type":"ContainerStarted","Data":"4aee02b45b6e93a5c24d8e354782f4f09d732839ec76c3a6cdfaf34a60f7b18e"}
May 28 17:46:05.151918 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:05.151846    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-t9dpl" podStartSLOduration=129.216968664 podStartE2EDuration="2m11.151831766s" podCreationTimestamp="2026-05-28 17:43:54 +0000 UTC" firstStartedPulling="2026-05-28 17:46:03.078343577 +0000 UTC m=+164.182039274" lastFinishedPulling="2026-05-28 17:46:05.013206676 +0000 UTC m=+166.116902376" observedRunningTime="2026-05-28 17:46:05.150625867 +0000 UTC m=+166.254321587" watchObservedRunningTime="2026-05-28 17:46:05.151831766 +0000 UTC m=+166.255527483"
May 28 17:46:06.141333 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:06.141300    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cf92m" event={"ID":"b8a6a83f-2e25-446a-8bac-f1c1e9f4defc","Type":"ContainerStarted","Data":"f5dfffb8ce48274ddd9288de60d86ee71880f1c94df699fe185ad8892f01eddb"}
May 28 17:46:06.141742 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:06.141404    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cf92m"
May 28 17:46:06.158104 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:06.158058    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-cf92m" podStartSLOduration=130.237823156 podStartE2EDuration="2m12.158047142s" podCreationTimestamp="2026-05-28 17:43:54 +0000 UTC" firstStartedPulling="2026-05-28 17:46:03.089217821 +0000 UTC m=+164.192913518" lastFinishedPulling="2026-05-28 17:46:05.009441808 +0000 UTC m=+166.113137504" observedRunningTime="2026-05-28 17:46:06.15725846 +0000 UTC m=+167.260954188" watchObservedRunningTime="2026-05-28 17:46:06.158047142 +0000 UTC m=+167.261742860"
May 28 17:46:11.514154 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:11.514121    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:46:15.867074 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:15.867030    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn" podUID="2b3a6a5a-c520-494b-8399-d47da5508054" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.12:8000/readyz\": dial tcp 10.132.0.12:8000: connect: connection refused"
May 28 17:46:16.145900 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.145822    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cf92m"
May 28 17:46:16.167962 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.167909    2560 generic.go:358] "Generic (PLEG): container finished" podID="2b3a6a5a-c520-494b-8399-d47da5508054" containerID="1cb7584e3e1bba75ea4e8e6128f5b3eced0464a1581cec38316b1387f208ef8d" exitCode=1
May 28 17:46:16.168121 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.167989    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn" event={"ID":"2b3a6a5a-c520-494b-8399-d47da5508054","Type":"ContainerDied","Data":"1cb7584e3e1bba75ea4e8e6128f5b3eced0464a1581cec38316b1387f208ef8d"}
May 28 17:46:16.168373 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.168350    2560 scope.go:117] "RemoveContainer" containerID="1cb7584e3e1bba75ea4e8e6128f5b3eced0464a1581cec38316b1387f208ef8d"
May 28 17:46:16.169278 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.169258    2560 generic.go:358] "Generic (PLEG): container finished" podID="ff40f739-b416-49f9-8397-9e1c54a743fc" containerID="07beb20c5088dd7a4a34de28afc971e01b31f9ceea1115a0630d37751fe23fb5" exitCode=255
May 28 17:46:16.169381 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.169291    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9" event={"ID":"ff40f739-b416-49f9-8397-9e1c54a743fc","Type":"ContainerDied","Data":"07beb20c5088dd7a4a34de28afc971e01b31f9ceea1115a0630d37751fe23fb5"}
May 28 17:46:16.169643 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.169582    2560 scope.go:117] "RemoveContainer" containerID="07beb20c5088dd7a4a34de28afc971e01b31f9ceea1115a0630d37751fe23fb5"
May 28 17:46:16.282960 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.282452    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9fg6h"]
May 28 17:46:16.286136 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.286112    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.289272 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.289102    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 28 17:46:16.289272 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.289108    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 28 17:46:16.289272 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.289157    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 28 17:46:16.289272 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.289268    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 28 17:46:16.289596 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.289410    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hm5hn\""
May 28 17:46:16.297469 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.297142    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9fg6h"]
May 28 17:46:16.387630 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.387592    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmhzq\" (UniqueName: \"kubernetes.io/projected/ae0d039a-6315-4200-a6a7-375dbc82f2c9-kube-api-access-lmhzq\") pod \"insights-runtime-extractor-9fg6h\" (UID: \"ae0d039a-6315-4200-a6a7-375dbc82f2c9\") " pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.387803 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.387643    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae0d039a-6315-4200-a6a7-375dbc82f2c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9fg6h\" (UID: \"ae0d039a-6315-4200-a6a7-375dbc82f2c9\") " pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.387803 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.387720    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ae0d039a-6315-4200-a6a7-375dbc82f2c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9fg6h\" (UID: \"ae0d039a-6315-4200-a6a7-375dbc82f2c9\") " pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.387803 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.387758    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae0d039a-6315-4200-a6a7-375dbc82f2c9-data-volume\") pod \"insights-runtime-extractor-9fg6h\" (UID: \"ae0d039a-6315-4200-a6a7-375dbc82f2c9\") " pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.387911 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.387836    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ae0d039a-6315-4200-a6a7-375dbc82f2c9-crio-socket\") pod \"insights-runtime-extractor-9fg6h\" (UID: \"ae0d039a-6315-4200-a6a7-375dbc82f2c9\") " pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.439670 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.439557    2560 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9"
May 28 17:46:16.472903 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.472865    2560 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"
May 28 17:46:16.488214 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.488183    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ae0d039a-6315-4200-a6a7-375dbc82f2c9-crio-socket\") pod \"insights-runtime-extractor-9fg6h\" (UID: \"ae0d039a-6315-4200-a6a7-375dbc82f2c9\") " pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.488364 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.488237    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lmhzq\" (UniqueName: \"kubernetes.io/projected/ae0d039a-6315-4200-a6a7-375dbc82f2c9-kube-api-access-lmhzq\") pod \"insights-runtime-extractor-9fg6h\" (UID: \"ae0d039a-6315-4200-a6a7-375dbc82f2c9\") " pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.488364 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.488266    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae0d039a-6315-4200-a6a7-375dbc82f2c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9fg6h\" (UID: \"ae0d039a-6315-4200-a6a7-375dbc82f2c9\") " pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.488364 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.488291    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ae0d039a-6315-4200-a6a7-375dbc82f2c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9fg6h\" (UID: \"ae0d039a-6315-4200-a6a7-375dbc82f2c9\") " pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.488364 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.488314    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae0d039a-6315-4200-a6a7-375dbc82f2c9-data-volume\") pod \"insights-runtime-extractor-9fg6h\" (UID: \"ae0d039a-6315-4200-a6a7-375dbc82f2c9\") " pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.488364 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.488308    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ae0d039a-6315-4200-a6a7-375dbc82f2c9-crio-socket\") pod \"insights-runtime-extractor-9fg6h\" (UID: \"ae0d039a-6315-4200-a6a7-375dbc82f2c9\") " pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.488771 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.488750    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ae0d039a-6315-4200-a6a7-375dbc82f2c9-data-volume\") pod \"insights-runtime-extractor-9fg6h\" (UID: \"ae0d039a-6315-4200-a6a7-375dbc82f2c9\") " pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.488907 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.488889    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ae0d039a-6315-4200-a6a7-375dbc82f2c9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9fg6h\" (UID: \"ae0d039a-6315-4200-a6a7-375dbc82f2c9\") " pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.490786 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.490765    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ae0d039a-6315-4200-a6a7-375dbc82f2c9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9fg6h\" (UID: \"ae0d039a-6315-4200-a6a7-375dbc82f2c9\") " pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.497072 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.497052    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmhzq\" (UniqueName: \"kubernetes.io/projected/ae0d039a-6315-4200-a6a7-375dbc82f2c9-kube-api-access-lmhzq\") pod \"insights-runtime-extractor-9fg6h\" (UID: \"ae0d039a-6315-4200-a6a7-375dbc82f2c9\") " pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.597998 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.597962    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9fg6h"
May 28 17:46:16.728444 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:16.728370    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9fg6h"]
May 28 17:46:16.731901 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:46:16.731871    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae0d039a_6315_4200_a6a7_375dbc82f2c9.slice/crio-d4b19f1a518cf85d40e6a8c06cf33162eb08a0ce1bb319e56adeaf7b3f41efbd WatchSource:0}: Error finding container d4b19f1a518cf85d40e6a8c06cf33162eb08a0ce1bb319e56adeaf7b3f41efbd: Status 404 returned error can't find the container with id d4b19f1a518cf85d40e6a8c06cf33162eb08a0ce1bb319e56adeaf7b3f41efbd
May 28 17:46:17.174073 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:17.174034    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn" event={"ID":"2b3a6a5a-c520-494b-8399-d47da5508054","Type":"ContainerStarted","Data":"72d7c71cdde4578d10ebe463d88c9c8f2ebbe4b829ff75a7f58b8fb16f4bcf3b"}
May 28 17:46:17.174553 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:17.174197    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"
May 28 17:46:17.175433 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:17.175405    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b6664dc8f-zxsjn"
May 28 17:46:17.175547 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:17.175516    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9fg6h" event={"ID":"ae0d039a-6315-4200-a6a7-375dbc82f2c9","Type":"ContainerStarted","Data":"64ed8bdec845bf9dc25e8ae40645991f45766041e05f091cdec82e27c71a02a1"}
May 28 17:46:17.175547 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:17.175539    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9fg6h" event={"ID":"ae0d039a-6315-4200-a6a7-375dbc82f2c9","Type":"ContainerStarted","Data":"d4b19f1a518cf85d40e6a8c06cf33162eb08a0ce1bb319e56adeaf7b3f41efbd"}
May 28 17:46:17.177054 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:17.177027    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f894b5f5b-rtvm9" event={"ID":"ff40f739-b416-49f9-8397-9e1c54a743fc","Type":"ContainerStarted","Data":"bbb0b995d25d2289e297d457b127acebb787ba900a935361daffc3087c5a2832"}
May 28 17:46:18.182459 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:18.182414    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9fg6h" event={"ID":"ae0d039a-6315-4200-a6a7-375dbc82f2c9","Type":"ContainerStarted","Data":"f2c8d0bdc4599e5bb2f18230c5fc5e1d3c3cd67f8999e793d8f32236bb978207"}
May 28 17:46:19.186517 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:19.186426    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9fg6h" event={"ID":"ae0d039a-6315-4200-a6a7-375dbc82f2c9","Type":"ContainerStarted","Data":"85adf9e9a0321f436698cd6b189860ee7c2b2a0771781016623b3b8157af7d73"}
May 28 17:46:19.203660 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:19.203611    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9fg6h" podStartSLOduration=1.154203458 podStartE2EDuration="3.203595329s" podCreationTimestamp="2026-05-28 17:46:16 +0000 UTC" firstStartedPulling="2026-05-28 17:46:16.794606496 +0000 UTC m=+177.898302193" lastFinishedPulling="2026-05-28 17:46:18.843998349 +0000 UTC m=+179.947694064" observedRunningTime="2026-05-28 17:46:19.202524079 +0000 UTC m=+180.306219837" watchObservedRunningTime="2026-05-28 17:46:19.203595329 +0000 UTC m=+180.307291046"
May 28 17:46:22.925525 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:22.925492    2560 patch_prober.go:28] interesting pod/image-registry-ffdf9ff96-wh6rt container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 28 17:46:22.925902 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:22.925541    2560 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt" podUID="909a7f25-5e01-448d-a17b-2eb5af1e5e5e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 28 17:46:25.141307 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:25.141279    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:46:27.184427 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.184390    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9b8kj"]
May 28 17:46:27.187947 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.187908    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.190815 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.190778    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 28 17:46:27.190815 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.190812    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sn8gc\""
May 28 17:46:27.191919 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.191899    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 28 17:46:27.192057 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.191957    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 28 17:46:27.192057 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.191990    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 28 17:46:27.192057 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.191997    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 28 17:46:27.192057 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.192045    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 28 17:46:27.265815 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.265775    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-root\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.265815 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.265817    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smmfl\" (UniqueName: \"kubernetes.io/projected/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-kube-api-access-smmfl\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.266071 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.265839    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-sys\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.266071 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.265887    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-accelerators-collector-config\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.266071 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.266001    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-tls\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.266071 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.266053    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-textfile\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.266225 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.266081    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-metrics-client-ca\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.266225 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.266102    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.266225 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.266161    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-wtmp\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.367040 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.367003    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-tls\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.367245 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.367079    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-textfile\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.367245 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.367110    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-metrics-client-ca\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.367245 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.367135    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.367245 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:46:27.367176    2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
May 28 17:46:27.367245 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.367200    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-wtmp\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.367245 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.367225    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-root\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.367509 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.367247    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-smmfl\" (UniqueName: \"kubernetes.io/projected/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-kube-api-access-smmfl\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.367509 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:46:27.367283    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-tls podName:3a8f9dc1-b903-4ee5-a69d-27f2c1d29320 nodeName:}" failed. No retries permitted until 2026-05-28 17:46:27.867261494 +0000 UTC m=+188.970957193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-tls") pod "node-exporter-9b8kj" (UID: "3a8f9dc1-b903-4ee5-a69d-27f2c1d29320") : secret "node-exporter-tls" not found
May 28 17:46:27.367509 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.367311    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-sys\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.367509 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.367317    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-root\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.367509 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.367351    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-accelerators-collector-config\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.367509 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.367383    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-wtmp\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.367509 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.367383    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-sys\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.367509 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.367496    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-textfile\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.367971 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.367951    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-accelerators-collector-config\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.368026 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.367970    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-metrics-client-ca\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.369713 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.369696    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.377635 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.377613    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smmfl\" (UniqueName: \"kubernetes.io/projected/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-kube-api-access-smmfl\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.871131 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.871085    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-tls\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:27.873509 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:27.873478    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3a8f9dc1-b903-4ee5-a69d-27f2c1d29320-node-exporter-tls\") pod \"node-exporter-9b8kj\" (UID: \"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320\") " pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:28.097247 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:28.097209    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9b8kj"
May 28 17:46:28.105627 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:46:28.105598    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a8f9dc1_b903_4ee5_a69d_27f2c1d29320.slice/crio-8f0a013156cb6a4eaa5b60e24b36b231fa94d5c25e41e78a93198428c800b493 WatchSource:0}: Error finding container 8f0a013156cb6a4eaa5b60e24b36b231fa94d5c25e41e78a93198428c800b493: Status 404 returned error can't find the container with id 8f0a013156cb6a4eaa5b60e24b36b231fa94d5c25e41e78a93198428c800b493
May 28 17:46:28.210885 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:28.210785    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9b8kj" event={"ID":"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320","Type":"ContainerStarted","Data":"8f0a013156cb6a4eaa5b60e24b36b231fa94d5c25e41e78a93198428c800b493"}
May 28 17:46:30.217683 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:30.217645    2560 generic.go:358] "Generic (PLEG): container finished" podID="3a8f9dc1-b903-4ee5-a69d-27f2c1d29320" containerID="54804f8b9f032b8fc11163f4b4bf727e5100e6837ef867c7e154ec3e695e5e61" exitCode=0
May 28 17:46:30.218083 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:30.217689    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9b8kj" event={"ID":"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320","Type":"ContainerDied","Data":"54804f8b9f032b8fc11163f4b4bf727e5100e6837ef867c7e154ec3e695e5e61"}
May 28 17:46:31.222497 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:31.222452    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9b8kj" event={"ID":"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320","Type":"ContainerStarted","Data":"83bbb9fe5486f88e704bd230d40f3b563a741bd6a66d40e013bfa9ea103fa871"}
May 28 17:46:31.222497 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:31.222502    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9b8kj" event={"ID":"3a8f9dc1-b903-4ee5-a69d-27f2c1d29320","Type":"ContainerStarted","Data":"ac19a0784e08b6ef591a2516195ff65e3fbe972904e1d28c4c3a97f31579b1e4"}
May 28 17:46:31.245071 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:31.245027    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9b8kj" podStartSLOduration=3.181846244 podStartE2EDuration="4.245013448s" podCreationTimestamp="2026-05-28 17:46:27 +0000 UTC" firstStartedPulling="2026-05-28 17:46:28.107467146 +0000 UTC m=+189.211162843" lastFinishedPulling="2026-05-28 17:46:29.170634352 +0000 UTC m=+190.274330047" observedRunningTime="2026-05-28 17:46:31.244131406 +0000 UTC m=+192.347827123" watchObservedRunningTime="2026-05-28 17:46:31.245013448 +0000 UTC m=+192.348709162"
May 28 17:46:39.725558 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:46:39.725523    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-ffdf9ff96-wh6rt"]
May 28 17:47:04.745264 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:04.745197    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt" podUID="909a7f25-5e01-448d-a17b-2eb5af1e5e5e" containerName="registry" containerID="cri-o://fb5c309385af9d9fa083876984723f6afe5076755581037a73bb64bbffe34eb4" gracePeriod=30
May 28 17:47:04.984702 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:04.984679    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:47:05.060685 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.060663    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-trusted-ca\") pod \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") "
May 28 17:47:05.060818 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.060701    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kgww\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-kube-api-access-4kgww\") pod \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") "
May 28 17:47:05.060818 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.060719    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-certificates\") pod \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") "
May 28 17:47:05.060818 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.060799    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls\") pod \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") "
May 28 17:47:05.060997 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.060888    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-image-registry-private-configuration\") pod \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") "
May 28 17:47:05.060997 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.060950    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-installation-pull-secrets\") pod \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") "
May 28 17:47:05.060997 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.060981    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-bound-sa-token\") pod \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") "
May 28 17:47:05.061153 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.061011    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-ca-trust-extracted\") pod \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\" (UID: \"909a7f25-5e01-448d-a17b-2eb5af1e5e5e\") "
May 28 17:47:05.061153 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.061115    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "909a7f25-5e01-448d-a17b-2eb5af1e5e5e" (UID: "909a7f25-5e01-448d-a17b-2eb5af1e5e5e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 17:47:05.061153 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.061126    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "909a7f25-5e01-448d-a17b-2eb5af1e5e5e" (UID: "909a7f25-5e01-448d-a17b-2eb5af1e5e5e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 17:47:05.061651 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.061593    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-trusted-ca\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:47:05.061651 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.061617    2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-certificates\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:47:05.063329 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.063284    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "909a7f25-5e01-448d-a17b-2eb5af1e5e5e" (UID: "909a7f25-5e01-448d-a17b-2eb5af1e5e5e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:47:05.063574 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.063547    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "909a7f25-5e01-448d-a17b-2eb5af1e5e5e" (UID: "909a7f25-5e01-448d-a17b-2eb5af1e5e5e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 17:47:05.063574 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.063551    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-kube-api-access-4kgww" (OuterVolumeSpecName: "kube-api-access-4kgww") pod "909a7f25-5e01-448d-a17b-2eb5af1e5e5e" (UID: "909a7f25-5e01-448d-a17b-2eb5af1e5e5e"). InnerVolumeSpecName "kube-api-access-4kgww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:47:05.063834 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.063814    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "909a7f25-5e01-448d-a17b-2eb5af1e5e5e" (UID: "909a7f25-5e01-448d-a17b-2eb5af1e5e5e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 17:47:05.063900 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.063857    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "909a7f25-5e01-448d-a17b-2eb5af1e5e5e" (UID: "909a7f25-5e01-448d-a17b-2eb5af1e5e5e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:47:05.069628 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.069605    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "909a7f25-5e01-448d-a17b-2eb5af1e5e5e" (UID: "909a7f25-5e01-448d-a17b-2eb5af1e5e5e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 17:47:05.162683 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.162659    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kgww\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-kube-api-access-4kgww\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:47:05.162683 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.162679    2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-registry-tls\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:47:05.162811 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.162689    2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-image-registry-private-configuration\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:47:05.162811 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.162698    2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-installation-pull-secrets\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:47:05.162811 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.162708    2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-bound-sa-token\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:47:05.162811 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.162716    2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/909a7f25-5e01-448d-a17b-2eb5af1e5e5e-ca-trust-extracted\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:47:05.311543 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.311460    2560 generic.go:358] "Generic (PLEG): container finished" podID="909a7f25-5e01-448d-a17b-2eb5af1e5e5e" containerID="fb5c309385af9d9fa083876984723f6afe5076755581037a73bb64bbffe34eb4" exitCode=0
May 28 17:47:05.311543 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.311525    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt"
May 28 17:47:05.311750 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.311553    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt" event={"ID":"909a7f25-5e01-448d-a17b-2eb5af1e5e5e","Type":"ContainerDied","Data":"fb5c309385af9d9fa083876984723f6afe5076755581037a73bb64bbffe34eb4"}
May 28 17:47:05.311750 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.311593    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-ffdf9ff96-wh6rt" event={"ID":"909a7f25-5e01-448d-a17b-2eb5af1e5e5e","Type":"ContainerDied","Data":"c403e52fd461533ba99eaafa331f4635b7421d8082279d313456af21c6cb327e"}
May 28 17:47:05.311750 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.311609    2560 scope.go:117] "RemoveContainer" containerID="fb5c309385af9d9fa083876984723f6afe5076755581037a73bb64bbffe34eb4"
May 28 17:47:05.319841 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.319825    2560 scope.go:117] "RemoveContainer" containerID="fb5c309385af9d9fa083876984723f6afe5076755581037a73bb64bbffe34eb4"
May 28 17:47:05.320153 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:47:05.320131    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb5c309385af9d9fa083876984723f6afe5076755581037a73bb64bbffe34eb4\": container with ID starting with fb5c309385af9d9fa083876984723f6afe5076755581037a73bb64bbffe34eb4 not found: ID does not exist" containerID="fb5c309385af9d9fa083876984723f6afe5076755581037a73bb64bbffe34eb4"
May 28 17:47:05.320233 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.320167    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb5c309385af9d9fa083876984723f6afe5076755581037a73bb64bbffe34eb4"} err="failed to get container status \"fb5c309385af9d9fa083876984723f6afe5076755581037a73bb64bbffe34eb4\": rpc error: code = NotFound desc = could not find container \"fb5c309385af9d9fa083876984723f6afe5076755581037a73bb64bbffe34eb4\": container with ID starting with fb5c309385af9d9fa083876984723f6afe5076755581037a73bb64bbffe34eb4 not found: ID does not exist"
May 28 17:47:05.330985 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.330959    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-ffdf9ff96-wh6rt"]
May 28 17:47:05.334753 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.334729    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-ffdf9ff96-wh6rt"]
May 28 17:47:05.521107 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:05.518378    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="909a7f25-5e01-448d-a17b-2eb5af1e5e5e" path="/var/lib/kubelet/pods/909a7f25-5e01-448d-a17b-2eb5af1e5e5e/volumes"
May 28 17:47:16.447504 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:16.447457    2560 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d" podUID="9c219cc6-46bf-4924-b97b-3285fb0d799b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 17:47:26.446657 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:26.446615    2560 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d" podUID="9c219cc6-46bf-4924-b97b-3285fb0d799b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 17:47:30.456394 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:30.456344    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs\") pod \"network-metrics-daemon-rh246\" (UID: \"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4\") " pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:47:30.458866 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:30.458833    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4693da93-c00a-4dd7-8dd0-fb3be0c73ad4-metrics-certs\") pod \"network-metrics-daemon-rh246\" (UID: \"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4\") " pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:47:30.716881 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:30.716800    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5wt76\""
May 28 17:47:30.724588 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:30.724566    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rh246"
May 28 17:47:30.844807 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:30.844780    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rh246"]
May 28 17:47:30.847229 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:47:30.847205    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4693da93_c00a_4dd7_8dd0_fb3be0c73ad4.slice/crio-bb937ea06cd686f0c68093f1b69831833f8f544cdef7550b6defab9f7d1aaf23 WatchSource:0}: Error finding container bb937ea06cd686f0c68093f1b69831833f8f544cdef7550b6defab9f7d1aaf23: Status 404 returned error can't find the container with id bb937ea06cd686f0c68093f1b69831833f8f544cdef7550b6defab9f7d1aaf23
May 28 17:47:31.381213 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:31.381168    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rh246" event={"ID":"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4","Type":"ContainerStarted","Data":"bb937ea06cd686f0c68093f1b69831833f8f544cdef7550b6defab9f7d1aaf23"}
May 28 17:47:32.385893 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:32.385858    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rh246" event={"ID":"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4","Type":"ContainerStarted","Data":"3b0f40b79e7c0073ec68905d1194b00a6ce8d933e2afcdf6deed557a4f69121e"}
May 28 17:47:32.385893 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:32.385892    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rh246" event={"ID":"4693da93-c00a-4dd7-8dd0-fb3be0c73ad4","Type":"ContainerStarted","Data":"781a2af06b9d38caa3618260e72cecad5c16c59985d545ed668627ebf8bfdd11"}
May 28 17:47:32.400166 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:32.400118    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rh246" podStartSLOduration=252.471513462 podStartE2EDuration="4m13.400104764s" podCreationTimestamp="2026-05-28 17:43:19 +0000 UTC" firstStartedPulling="2026-05-28 17:47:30.849156609 +0000 UTC m=+251.952852309" lastFinishedPulling="2026-05-28 17:47:31.777747916 +0000 UTC m=+252.881443611" observedRunningTime="2026-05-28 17:47:32.398993224 +0000 UTC m=+253.502688941" watchObservedRunningTime="2026-05-28 17:47:32.400104764 +0000 UTC m=+253.503800514"
May 28 17:47:36.447180 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:36.447100    2560 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d" podUID="9c219cc6-46bf-4924-b97b-3285fb0d799b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 17:47:36.447180 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:36.447163    2560 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d"
May 28 17:47:36.447672 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:36.447607    2560 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"83dcde622c6fc59cb31b5aac017841a9650944fcb7f35770fc193537425eeaab"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d" containerMessage="Container service-proxy failed liveness probe, will be restarted"
May 28 17:47:36.447672 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:36.447644    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d" podUID="9c219cc6-46bf-4924-b97b-3285fb0d799b" containerName="service-proxy" containerID="cri-o://83dcde622c6fc59cb31b5aac017841a9650944fcb7f35770fc193537425eeaab" gracePeriod=30
May 28 17:47:37.400503 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:37.400470    2560 generic.go:358] "Generic (PLEG): container finished" podID="9c219cc6-46bf-4924-b97b-3285fb0d799b" containerID="83dcde622c6fc59cb31b5aac017841a9650944fcb7f35770fc193537425eeaab" exitCode=2
May 28 17:47:37.400674 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:37.400538    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d" event={"ID":"9c219cc6-46bf-4924-b97b-3285fb0d799b","Type":"ContainerDied","Data":"83dcde622c6fc59cb31b5aac017841a9650944fcb7f35770fc193537425eeaab"}
May 28 17:47:37.400674 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:47:37.400575    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cf5d84b67-zd86d" event={"ID":"9c219cc6-46bf-4924-b97b-3285fb0d799b","Type":"ContainerStarted","Data":"c72f95cf0a81b14145b74b1e0dfda391e5cf553d602420a46f04fa3d73d3bd87"}
May 28 17:48:01.110667 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:48:01.110619    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps" podUID="655375f3-42df-4046-b86a-386dfee4ce6d"
May 28 17:48:01.466831 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:48:01.466747    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:48:04.816429 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:48:04.816383    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-27wps\" (UID: \"655375f3-42df-4046-b86a-386dfee4ce6d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:48:04.818852 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:48:04.818820    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/655375f3-42df-4046-b86a-386dfee4ce6d-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-27wps\" (UID: \"655375f3-42df-4046-b86a-386dfee4ce6d\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:48:05.069723 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:48:05.069625    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9mcx8\""
May 28 17:48:05.077756 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:48:05.077724    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"
May 28 17:48:05.230905 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:48:05.230869    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-27wps"]
May 28 17:48:05.234197 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:48:05.234167    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod655375f3_42df_4046_b86a_386dfee4ce6d.slice/crio-db9fb1f421f34ace338b2ce56d771791b6ac5569c3034ff66f24387e18ed8e7c WatchSource:0}: Error finding container db9fb1f421f34ace338b2ce56d771791b6ac5569c3034ff66f24387e18ed8e7c: Status 404 returned error can't find the container with id db9fb1f421f34ace338b2ce56d771791b6ac5569c3034ff66f24387e18ed8e7c
May 28 17:48:05.479189 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:48:05.479095    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps" event={"ID":"655375f3-42df-4046-b86a-386dfee4ce6d","Type":"ContainerStarted","Data":"db9fb1f421f34ace338b2ce56d771791b6ac5569c3034ff66f24387e18ed8e7c"}
May 28 17:48:06.486059 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:48:06.486022    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps" event={"ID":"655375f3-42df-4046-b86a-386dfee4ce6d","Type":"ContainerStarted","Data":"86898e61cf1b7c256955461943ad5db85e3fa9d6d3d77fc127bcc91960bad6af"}
May 28 17:48:06.502509 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:48:06.502459    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-27wps" podStartSLOduration=277.431276433 podStartE2EDuration="4m38.502443324s" podCreationTimestamp="2026-05-28 17:43:28 +0000 UTC" firstStartedPulling="2026-05-28 17:48:05.235981403 +0000 UTC m=+286.339677100" lastFinishedPulling="2026-05-28 17:48:06.30714826 +0000 UTC m=+287.410843991" observedRunningTime="2026-05-28 17:48:06.501261808 +0000 UTC m=+287.604957526" watchObservedRunningTime="2026-05-28 17:48:06.502443324 +0000 UTC m=+287.606139035"
May 28 17:48:19.419067 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:48:19.419036    2560 kubelet.go:1628] "Image garbage collection succeeded"
May 28 17:50:57.347877 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.347799    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-xtdb5"]
May 28 17:50:57.348333 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.348086    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="909a7f25-5e01-448d-a17b-2eb5af1e5e5e" containerName="registry"
May 28 17:50:57.348333 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.348100    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="909a7f25-5e01-448d-a17b-2eb5af1e5e5e" containerName="registry"
May 28 17:50:57.348333 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.348152    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="909a7f25-5e01-448d-a17b-2eb5af1e5e5e" containerName="registry"
May 28 17:50:57.350849 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.350832    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-xtdb5"
May 28 17:50:57.353013 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.352993    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
May 28 17:50:57.354012 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.353993    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-5mjzp\""
May 28 17:50:57.354074 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.354047    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
May 28 17:50:57.359960 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.359918    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-xtdb5"]
May 28 17:50:57.412611 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.412584    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvbk6\" (UniqueName: \"kubernetes.io/projected/442271d6-e2a6-4624-9874-96dac5851140-kube-api-access-xvbk6\") pod \"cert-manager-79c8d999ff-xtdb5\" (UID: \"442271d6-e2a6-4624-9874-96dac5851140\") " pod="cert-manager/cert-manager-79c8d999ff-xtdb5"
May 28 17:50:57.412706 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.412623    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/442271d6-e2a6-4624-9874-96dac5851140-bound-sa-token\") pod \"cert-manager-79c8d999ff-xtdb5\" (UID: \"442271d6-e2a6-4624-9874-96dac5851140\") " pod="cert-manager/cert-manager-79c8d999ff-xtdb5"
May 28 17:50:57.513411 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.513384    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xvbk6\" (UniqueName: \"kubernetes.io/projected/442271d6-e2a6-4624-9874-96dac5851140-kube-api-access-xvbk6\") pod \"cert-manager-79c8d999ff-xtdb5\" (UID: \"442271d6-e2a6-4624-9874-96dac5851140\") " pod="cert-manager/cert-manager-79c8d999ff-xtdb5"
May 28 17:50:57.513490 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.513426    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/442271d6-e2a6-4624-9874-96dac5851140-bound-sa-token\") pod \"cert-manager-79c8d999ff-xtdb5\" (UID: \"442271d6-e2a6-4624-9874-96dac5851140\") " pod="cert-manager/cert-manager-79c8d999ff-xtdb5"
May 28 17:50:57.521767 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.521727    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/442271d6-e2a6-4624-9874-96dac5851140-bound-sa-token\") pod \"cert-manager-79c8d999ff-xtdb5\" (UID: \"442271d6-e2a6-4624-9874-96dac5851140\") " pod="cert-manager/cert-manager-79c8d999ff-xtdb5"
May 28 17:50:57.521870 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.521833    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvbk6\" (UniqueName: \"kubernetes.io/projected/442271d6-e2a6-4624-9874-96dac5851140-kube-api-access-xvbk6\") pod \"cert-manager-79c8d999ff-xtdb5\" (UID: \"442271d6-e2a6-4624-9874-96dac5851140\") " pod="cert-manager/cert-manager-79c8d999ff-xtdb5"
May 28 17:50:57.660528 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.660463    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-xtdb5"
May 28 17:50:57.788638 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.788611    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-xtdb5"]
May 28 17:50:57.792108 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:50:57.792081    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod442271d6_e2a6_4624_9874_96dac5851140.slice/crio-6eab88f835e1771ea2878a71fc8d6bac7959ff88e9c6dbde1a8bede6a6e859ca WatchSource:0}: Error finding container 6eab88f835e1771ea2878a71fc8d6bac7959ff88e9c6dbde1a8bede6a6e859ca: Status 404 returned error can't find the container with id 6eab88f835e1771ea2878a71fc8d6bac7959ff88e9c6dbde1a8bede6a6e859ca
May 28 17:50:57.794278 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.794261    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 17:50:57.934967 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:50:57.934886    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-xtdb5" event={"ID":"442271d6-e2a6-4624-9874-96dac5851140","Type":"ContainerStarted","Data":"6eab88f835e1771ea2878a71fc8d6bac7959ff88e9c6dbde1a8bede6a6e859ca"}
May 28 17:51:00.945007 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:00.944965    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-xtdb5" event={"ID":"442271d6-e2a6-4624-9874-96dac5851140","Type":"ContainerStarted","Data":"8e8308299055667840e6feb5bee3eeb545ef3481d5f2902db8ad68b9f480e8d0"}
May 28 17:51:00.960897 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:00.960840    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-xtdb5" podStartSLOduration=1.301781723 podStartE2EDuration="3.960825976s" podCreationTimestamp="2026-05-28 17:50:57 +0000 UTC" firstStartedPulling="2026-05-28 17:50:57.794386662 +0000 UTC m=+458.898082361" lastFinishedPulling="2026-05-28 17:51:00.453430918 +0000 UTC m=+461.557126614" observedRunningTime="2026-05-28 17:51:00.959856771 +0000 UTC m=+462.063552490" watchObservedRunningTime="2026-05-28 17:51:00.960825976 +0000 UTC m=+462.064521693"
May 28 17:51:05.566431 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.566395    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"]
May 28 17:51:05.569393 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.569374    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"
May 28 17:51:05.577002 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.576978    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
May 28 17:51:05.577149 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.577127    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
May 28 17:51:05.577234 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.577140    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
May 28 17:51:05.578045 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.578027    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
May 28 17:51:05.578203 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.578188    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-gdvhp\""
May 28 17:51:05.578247 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.578219    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
May 28 17:51:05.599742 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.599715    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"]
May 28 17:51:05.665491 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.665466    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/48249e43-e627-467c-bc0f-76c2fd301a55-metrics-cert\") pod \"lws-controller-manager-655c6bbdd5-jtvwd\" (UID: \"48249e43-e627-467c-bc0f-76c2fd301a55\") " pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"
May 28 17:51:05.665591 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.665504    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ncbj\" (UniqueName: \"kubernetes.io/projected/48249e43-e627-467c-bc0f-76c2fd301a55-kube-api-access-4ncbj\") pod \"lws-controller-manager-655c6bbdd5-jtvwd\" (UID: \"48249e43-e627-467c-bc0f-76c2fd301a55\") " pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"
May 28 17:51:05.665591 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.665528    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/48249e43-e627-467c-bc0f-76c2fd301a55-manager-config\") pod \"lws-controller-manager-655c6bbdd5-jtvwd\" (UID: \"48249e43-e627-467c-bc0f-76c2fd301a55\") " pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"
May 28 17:51:05.665694 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.665616    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/48249e43-e627-467c-bc0f-76c2fd301a55-cert\") pod \"lws-controller-manager-655c6bbdd5-jtvwd\" (UID: \"48249e43-e627-467c-bc0f-76c2fd301a55\") " pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"
May 28 17:51:05.766545 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.766511    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/48249e43-e627-467c-bc0f-76c2fd301a55-cert\") pod \"lws-controller-manager-655c6bbdd5-jtvwd\" (UID: \"48249e43-e627-467c-bc0f-76c2fd301a55\") " pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"
May 28 17:51:05.766656 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.766556    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/48249e43-e627-467c-bc0f-76c2fd301a55-metrics-cert\") pod \"lws-controller-manager-655c6bbdd5-jtvwd\" (UID: \"48249e43-e627-467c-bc0f-76c2fd301a55\") " pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"
May 28 17:51:05.766718 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.766688    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4ncbj\" (UniqueName: \"kubernetes.io/projected/48249e43-e627-467c-bc0f-76c2fd301a55-kube-api-access-4ncbj\") pod \"lws-controller-manager-655c6bbdd5-jtvwd\" (UID: \"48249e43-e627-467c-bc0f-76c2fd301a55\") " pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"
May 28 17:51:05.766786 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.766722    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/48249e43-e627-467c-bc0f-76c2fd301a55-manager-config\") pod \"lws-controller-manager-655c6bbdd5-jtvwd\" (UID: \"48249e43-e627-467c-bc0f-76c2fd301a55\") " pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"
May 28 17:51:05.767389 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.767369    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/48249e43-e627-467c-bc0f-76c2fd301a55-manager-config\") pod \"lws-controller-manager-655c6bbdd5-jtvwd\" (UID: \"48249e43-e627-467c-bc0f-76c2fd301a55\") " pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"
May 28 17:51:05.769159 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.769140    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/48249e43-e627-467c-bc0f-76c2fd301a55-cert\") pod \"lws-controller-manager-655c6bbdd5-jtvwd\" (UID: \"48249e43-e627-467c-bc0f-76c2fd301a55\") " pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"
May 28 17:51:05.769344 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.769327    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/48249e43-e627-467c-bc0f-76c2fd301a55-metrics-cert\") pod \"lws-controller-manager-655c6bbdd5-jtvwd\" (UID: \"48249e43-e627-467c-bc0f-76c2fd301a55\") " pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"
May 28 17:51:05.780306 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.780278    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ncbj\" (UniqueName: \"kubernetes.io/projected/48249e43-e627-467c-bc0f-76c2fd301a55-kube-api-access-4ncbj\") pod \"lws-controller-manager-655c6bbdd5-jtvwd\" (UID: \"48249e43-e627-467c-bc0f-76c2fd301a55\") " pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"
May 28 17:51:05.879243 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:05.879189    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"
May 28 17:51:06.000380 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:06.000358    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"]
May 28 17:51:06.003251 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:51:06.003228    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48249e43_e627_467c_bc0f_76c2fd301a55.slice/crio-72ed66a69ac9a5c6d37f89b2bd64039068865ab16e397459d19cc9c6f0e1fb58 WatchSource:0}: Error finding container 72ed66a69ac9a5c6d37f89b2bd64039068865ab16e397459d19cc9c6f0e1fb58: Status 404 returned error can't find the container with id 72ed66a69ac9a5c6d37f89b2bd64039068865ab16e397459d19cc9c6f0e1fb58
May 28 17:51:06.963375 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:06.963336    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd" event={"ID":"48249e43-e627-467c-bc0f-76c2fd301a55","Type":"ContainerStarted","Data":"72ed66a69ac9a5c6d37f89b2bd64039068865ab16e397459d19cc9c6f0e1fb58"}
May 28 17:51:08.971150 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:08.971111    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd" event={"ID":"48249e43-e627-467c-bc0f-76c2fd301a55","Type":"ContainerStarted","Data":"e2e6f3e3c90ec94a9104bf607ae30144a4f1b40024f97bbd23f9d63d54521c51"}
May 28 17:51:08.971518 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:08.971209    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"
May 28 17:51:08.991707 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:08.991663    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd" podStartSLOduration=1.483908652 podStartE2EDuration="3.991650147s" podCreationTimestamp="2026-05-28 17:51:05 +0000 UTC" firstStartedPulling="2026-05-28 17:51:06.004883337 +0000 UTC m=+467.108579032" lastFinishedPulling="2026-05-28 17:51:08.512624827 +0000 UTC m=+469.616320527" observedRunningTime="2026-05-28 17:51:08.990978858 +0000 UTC m=+470.094674578" watchObservedRunningTime="2026-05-28 17:51:08.991650147 +0000 UTC m=+470.095345903"
May 28 17:51:14.588781 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.588739    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8"]
May 28 17:51:14.592045 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.592018    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8"
May 28 17:51:14.594978 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.594950    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
May 28 17:51:14.595158 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.595138    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
May 28 17:51:14.595303 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.595279    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bpqhj\""
May 28 17:51:14.595714 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.595695    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
May 28 17:51:14.595788 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.595770    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
May 28 17:51:14.616544 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.616513    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8"]
May 28 17:51:14.726934 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.726910    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0970116a-e87a-4db0-8dd2-d2eb5f64f236-webhook-cert\") pod \"opendatahub-operator-controller-manager-65fbbfd459-2fsr8\" (UID: \"0970116a-e87a-4db0-8dd2-d2eb5f64f236\") " pod="opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8"
May 28 17:51:14.727027 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.726981    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v5pk\" (UniqueName: \"kubernetes.io/projected/0970116a-e87a-4db0-8dd2-d2eb5f64f236-kube-api-access-9v5pk\") pod \"opendatahub-operator-controller-manager-65fbbfd459-2fsr8\" (UID: \"0970116a-e87a-4db0-8dd2-d2eb5f64f236\") " pod="opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8"
May 28 17:51:14.727027 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.727006    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0970116a-e87a-4db0-8dd2-d2eb5f64f236-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65fbbfd459-2fsr8\" (UID: \"0970116a-e87a-4db0-8dd2-d2eb5f64f236\") " pod="opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8"
May 28 17:51:14.827522 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.827498    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0970116a-e87a-4db0-8dd2-d2eb5f64f236-webhook-cert\") pod \"opendatahub-operator-controller-manager-65fbbfd459-2fsr8\" (UID: \"0970116a-e87a-4db0-8dd2-d2eb5f64f236\") " pod="opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8"
May 28 17:51:14.827611 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.827549    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9v5pk\" (UniqueName: \"kubernetes.io/projected/0970116a-e87a-4db0-8dd2-d2eb5f64f236-kube-api-access-9v5pk\") pod \"opendatahub-operator-controller-manager-65fbbfd459-2fsr8\" (UID: \"0970116a-e87a-4db0-8dd2-d2eb5f64f236\") " pod="opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8"
May 28 17:51:14.827611 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.827586    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0970116a-e87a-4db0-8dd2-d2eb5f64f236-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65fbbfd459-2fsr8\" (UID: \"0970116a-e87a-4db0-8dd2-d2eb5f64f236\") " pod="opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8"
May 28 17:51:14.830333 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.830297    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0970116a-e87a-4db0-8dd2-d2eb5f64f236-webhook-cert\") pod \"opendatahub-operator-controller-manager-65fbbfd459-2fsr8\" (UID: \"0970116a-e87a-4db0-8dd2-d2eb5f64f236\") " pod="opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8"
May 28 17:51:14.830433 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.830383    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0970116a-e87a-4db0-8dd2-d2eb5f64f236-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65fbbfd459-2fsr8\" (UID: \"0970116a-e87a-4db0-8dd2-d2eb5f64f236\") " pod="opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8"
May 28 17:51:14.837383 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.837350    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v5pk\" (UniqueName: \"kubernetes.io/projected/0970116a-e87a-4db0-8dd2-d2eb5f64f236-kube-api-access-9v5pk\") pod \"opendatahub-operator-controller-manager-65fbbfd459-2fsr8\" (UID: \"0970116a-e87a-4db0-8dd2-d2eb5f64f236\") " pod="opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8"
May 28 17:51:14.903412 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:14.903353    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8"
May 28 17:51:15.027627 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:15.027598    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8"]
May 28 17:51:15.030737 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:51:15.030712    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0970116a_e87a_4db0_8dd2_d2eb5f64f236.slice/crio-17c6b2d6a1d41664301f530891018676fc4227af67a34d489efb3ff887a52910 WatchSource:0}: Error finding container 17c6b2d6a1d41664301f530891018676fc4227af67a34d489efb3ff887a52910: Status 404 returned error can't find the container with id 17c6b2d6a1d41664301f530891018676fc4227af67a34d489efb3ff887a52910
May 28 17:51:15.991418 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:15.991375    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8" event={"ID":"0970116a-e87a-4db0-8dd2-d2eb5f64f236","Type":"ContainerStarted","Data":"17c6b2d6a1d41664301f530891018676fc4227af67a34d489efb3ff887a52910"}
May 28 17:51:18.002878 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:18.002842    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8" event={"ID":"0970116a-e87a-4db0-8dd2-d2eb5f64f236","Type":"ContainerStarted","Data":"df039d7c11b7d3c3644274ed38facfd6e0aa58dbf13c50f5b116b3df25dd2625"}
May 28 17:51:18.003338 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:18.003045    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8"
May 28 17:51:18.030641 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:18.030242    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8" podStartSLOduration=1.6280253089999999 podStartE2EDuration="4.030225996s" podCreationTimestamp="2026-05-28 17:51:14 +0000 UTC" firstStartedPulling="2026-05-28 17:51:15.032291036 +0000 UTC m=+476.135986731" lastFinishedPulling="2026-05-28 17:51:17.434491719 +0000 UTC m=+478.538187418" observedRunningTime="2026-05-28 17:51:18.028346496 +0000 UTC m=+479.132042214" watchObservedRunningTime="2026-05-28 17:51:18.030225996 +0000 UTC m=+479.133921715"
May 28 17:51:19.976163 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:19.976136    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-655c6bbdd5-jtvwd"
May 28 17:51:29.007814 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:51:29.007774    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65fbbfd459-2fsr8"
May 28 17:52:07.774783 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.774686    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"]
May 28 17:52:07.777904 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.777884    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.780483 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.780459    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
May 28 17:52:07.780604 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.780465    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
May 28 17:52:07.780666 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.780625    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
May 28 17:52:07.780666 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.780646    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-5qwws\""
May 28 17:52:07.788447 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.788419    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"]
May 28 17:52:07.789619 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.789595    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33a6915a-2203-4ece-92ff-235f6e7f826b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.789719 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.789638    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33a6915a-2203-4ece-92ff-235f6e7f826b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.789719 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.789675    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j992\" (UniqueName: \"kubernetes.io/projected/33a6915a-2203-4ece-92ff-235f6e7f826b-kube-api-access-8j992\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.789834 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.789729    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a6915a-2203-4ece-92ff-235f6e7f826b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.789834 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.789761    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33a6915a-2203-4ece-92ff-235f6e7f826b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.789834 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.789789    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33a6915a-2203-4ece-92ff-235f6e7f826b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.789834 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.789823    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a6915a-2203-4ece-92ff-235f6e7f826b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.790017 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.789876    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33a6915a-2203-4ece-92ff-235f6e7f826b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.790017 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.789909    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33a6915a-2203-4ece-92ff-235f6e7f826b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.891016 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.890975    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33a6915a-2203-4ece-92ff-235f6e7f826b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.891016 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.891018    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33a6915a-2203-4ece-92ff-235f6e7f826b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.891239 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.891041    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a6915a-2203-4ece-92ff-235f6e7f826b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.891239 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.891070    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33a6915a-2203-4ece-92ff-235f6e7f826b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.891239 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.891194    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33a6915a-2203-4ece-92ff-235f6e7f826b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.891396 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.891259    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33a6915a-2203-4ece-92ff-235f6e7f826b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.891396 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.891317    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33a6915a-2203-4ece-92ff-235f6e7f826b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.891396 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.891363    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8j992\" (UniqueName: \"kubernetes.io/projected/33a6915a-2203-4ece-92ff-235f6e7f826b-kube-api-access-8j992\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.891551 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.891409    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a6915a-2203-4ece-92ff-235f6e7f826b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.891604 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.891554    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a6915a-2203-4ece-92ff-235f6e7f826b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.891655 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.891626    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33a6915a-2203-4ece-92ff-235f6e7f826b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.891758 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.891735    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33a6915a-2203-4ece-92ff-235f6e7f826b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.891825 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.891789    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33a6915a-2203-4ece-92ff-235f6e7f826b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.891825 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.891787    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33a6915a-2203-4ece-92ff-235f6e7f826b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.893798 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.893777    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33a6915a-2203-4ece-92ff-235f6e7f826b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.893982 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.893963    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33a6915a-2203-4ece-92ff-235f6e7f826b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.898831 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.898802    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33a6915a-2203-4ece-92ff-235f6e7f826b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:07.898831 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:07.898823    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j992\" (UniqueName: \"kubernetes.io/projected/33a6915a-2203-4ece-92ff-235f6e7f826b-kube-api-access-8j992\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4b424c8\" (UID: \"33a6915a-2203-4ece-92ff-235f6e7f826b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:08.089860 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:08.089830    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:08.219144 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:08.219109    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"]
May 28 17:52:08.222425 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:52:08.222389    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33a6915a_2203_4ece_92ff_235f6e7f826b.slice/crio-4bd1b6f6db8d93eee4dce269efdabfed7678a73d327cd3b1a9e5360c3915a45a WatchSource:0}: Error finding container 4bd1b6f6db8d93eee4dce269efdabfed7678a73d327cd3b1a9e5360c3915a45a: Status 404 returned error can't find the container with id 4bd1b6f6db8d93eee4dce269efdabfed7678a73d327cd3b1a9e5360c3915a45a
May 28 17:52:09.138413 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:09.138358    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8" event={"ID":"33a6915a-2203-4ece-92ff-235f6e7f826b","Type":"ContainerStarted","Data":"4bd1b6f6db8d93eee4dce269efdabfed7678a73d327cd3b1a9e5360c3915a45a"}
May 28 17:52:10.628919 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:10.628874    2560 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
May 28 17:52:10.629311 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:10.628983    2560 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
May 28 17:52:10.629311 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:10.629035    2560 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
May 28 17:52:11.145579 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:11.145532    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8" event={"ID":"33a6915a-2203-4ece-92ff-235f6e7f826b","Type":"ContainerStarted","Data":"1bce32500780d33c0887f9521009969c631bc2360e082abd62c7bbaf99357e97"}
May 28 17:52:11.172087 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:11.172031    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8" podStartSLOduration=1.767405431 podStartE2EDuration="4.172012756s" podCreationTimestamp="2026-05-28 17:52:07 +0000 UTC" firstStartedPulling="2026-05-28 17:52:08.224040903 +0000 UTC m=+529.327736599" lastFinishedPulling="2026-05-28 17:52:10.628648228 +0000 UTC m=+531.732343924" observedRunningTime="2026-05-28 17:52:11.170827995 +0000 UTC m=+532.274523710" watchObservedRunningTime="2026-05-28 17:52:11.172012756 +0000 UTC m=+532.275708474"
May 28 17:52:12.090875 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:12.090829    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:12.092199 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:12.092176    2560 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-686dffbf4b424c8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.17:15021/healthz/ready\": dial tcp 10.132.0.17:15021: connect: connection refused" start-of-body=
May 28 17:52:12.092248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:12.092232    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8" podUID="33a6915a-2203-4ece-92ff-235f6e7f826b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.17:15021/healthz/ready\": dial tcp 10.132.0.17:15021: connect: connection refused"
May 28 17:52:13.093991 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:13.093958    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:13.150752 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:13.150729    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:13.151478 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:13.151459    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4b424c8"
May 28 17:52:33.463990 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:33.463949    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4l4g7"]
May 28 17:52:33.467387 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:33.467367    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4l4g7"
May 28 17:52:33.470492 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:33.470465    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-rdwtx\""
May 28 17:52:33.471496 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:33.471472    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
May 28 17:52:33.471593 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:33.471520    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
May 28 17:52:33.477881 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:33.477855    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4l4g7"]
May 28 17:52:33.574508 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:33.574471    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q64wc\" (UniqueName: \"kubernetes.io/projected/56bdf272-85b8-4330-be08-e1efafdc3852-kube-api-access-q64wc\") pod \"kuadrant-operator-catalog-4l4g7\" (UID: \"56bdf272-85b8-4330-be08-e1efafdc3852\") " pod="kuadrant-system/kuadrant-operator-catalog-4l4g7"
May 28 17:52:33.675757 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:33.675719    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q64wc\" (UniqueName: \"kubernetes.io/projected/56bdf272-85b8-4330-be08-e1efafdc3852-kube-api-access-q64wc\") pod \"kuadrant-operator-catalog-4l4g7\" (UID: \"56bdf272-85b8-4330-be08-e1efafdc3852\") " pod="kuadrant-system/kuadrant-operator-catalog-4l4g7"
May 28 17:52:33.684901 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:33.684864    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q64wc\" (UniqueName: \"kubernetes.io/projected/56bdf272-85b8-4330-be08-e1efafdc3852-kube-api-access-q64wc\") pod \"kuadrant-operator-catalog-4l4g7\" (UID: \"56bdf272-85b8-4330-be08-e1efafdc3852\") " pod="kuadrant-system/kuadrant-operator-catalog-4l4g7"
May 28 17:52:33.776757 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:33.776725    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4l4g7"
May 28 17:52:33.812909 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:33.812862    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4l4g7"]
May 28 17:52:33.907639 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:33.907599    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4l4g7"]
May 28 17:52:33.910587 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:52:33.910545    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56bdf272_85b8_4330_be08_e1efafdc3852.slice/crio-97f9bc0ad8310de9db14dcae5862394aefaadd77038b39c6e86dfbefa3338828 WatchSource:0}: Error finding container 97f9bc0ad8310de9db14dcae5862394aefaadd77038b39c6e86dfbefa3338828: Status 404 returned error can't find the container with id 97f9bc0ad8310de9db14dcae5862394aefaadd77038b39c6e86dfbefa3338828
May 28 17:52:34.014200 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:34.014166    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-97fp7"]
May 28 17:52:34.018668 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:34.018650    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-97fp7"
May 28 17:52:34.026093 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:34.026066    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-97fp7"]
May 28 17:52:34.079074 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:34.078975    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv8gx\" (UniqueName: \"kubernetes.io/projected/039c2a50-474c-425d-9794-70ac390c81da-kube-api-access-wv8gx\") pod \"kuadrant-operator-catalog-97fp7\" (UID: \"039c2a50-474c-425d-9794-70ac390c81da\") " pod="kuadrant-system/kuadrant-operator-catalog-97fp7"
May 28 17:52:34.179848 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:34.179803    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wv8gx\" (UniqueName: \"kubernetes.io/projected/039c2a50-474c-425d-9794-70ac390c81da-kube-api-access-wv8gx\") pod \"kuadrant-operator-catalog-97fp7\" (UID: \"039c2a50-474c-425d-9794-70ac390c81da\") " pod="kuadrant-system/kuadrant-operator-catalog-97fp7"
May 28 17:52:34.190090 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:34.190059    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv8gx\" (UniqueName: \"kubernetes.io/projected/039c2a50-474c-425d-9794-70ac390c81da-kube-api-access-wv8gx\") pod \"kuadrant-operator-catalog-97fp7\" (UID: \"039c2a50-474c-425d-9794-70ac390c81da\") " pod="kuadrant-system/kuadrant-operator-catalog-97fp7"
May 28 17:52:34.210508 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:34.210471    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4l4g7" event={"ID":"56bdf272-85b8-4330-be08-e1efafdc3852","Type":"ContainerStarted","Data":"97f9bc0ad8310de9db14dcae5862394aefaadd77038b39c6e86dfbefa3338828"}
May 28 17:52:34.328356 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:34.328315    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-97fp7"
May 28 17:52:34.461184 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:34.461145    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-97fp7"]
May 28 17:52:34.464317 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:52:34.464288    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039c2a50_474c_425d_9794_70ac390c81da.slice/crio-c78facbb214514c61f087e6d95efaf715a4561166ba4625c5a99cb836cf55b3e WatchSource:0}: Error finding container c78facbb214514c61f087e6d95efaf715a4561166ba4625c5a99cb836cf55b3e: Status 404 returned error can't find the container with id c78facbb214514c61f087e6d95efaf715a4561166ba4625c5a99cb836cf55b3e
May 28 17:52:35.215715 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:35.215656    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-97fp7" event={"ID":"039c2a50-474c-425d-9794-70ac390c81da","Type":"ContainerStarted","Data":"c78facbb214514c61f087e6d95efaf715a4561166ba4625c5a99cb836cf55b3e"}
May 28 17:52:37.225497 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:37.225459    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4l4g7" event={"ID":"56bdf272-85b8-4330-be08-e1efafdc3852","Type":"ContainerStarted","Data":"9a19a401c3c099dd22955c7f8466e10bfdf239c745354dfe4826bea7799f42ef"}
May 28 17:52:37.226007 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:37.225567    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-4l4g7" podUID="56bdf272-85b8-4330-be08-e1efafdc3852" containerName="registry-server" containerID="cri-o://9a19a401c3c099dd22955c7f8466e10bfdf239c745354dfe4826bea7799f42ef" gracePeriod=2
May 28 17:52:37.227038 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:37.227014    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-97fp7" event={"ID":"039c2a50-474c-425d-9794-70ac390c81da","Type":"ContainerStarted","Data":"85d6560ec826aefb5d37484f373941039f28868e1f53e37b3246fcf1b7f51134"}
May 28 17:52:37.263265 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:37.263216    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4l4g7" podStartSLOduration=1.9134200319999999 podStartE2EDuration="4.263202309s" podCreationTimestamp="2026-05-28 17:52:33 +0000 UTC" firstStartedPulling="2026-05-28 17:52:33.912240995 +0000 UTC m=+555.015936691" lastFinishedPulling="2026-05-28 17:52:36.262023268 +0000 UTC m=+557.365718968" observedRunningTime="2026-05-28 17:52:37.244363346 +0000 UTC m=+558.348059065" watchObservedRunningTime="2026-05-28 17:52:37.263202309 +0000 UTC m=+558.366898025"
May 28 17:52:37.263420 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:37.263295    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-97fp7" podStartSLOduration=2.463413193 podStartE2EDuration="4.263290292s" podCreationTimestamp="2026-05-28 17:52:33 +0000 UTC" firstStartedPulling="2026-05-28 17:52:34.465641311 +0000 UTC m=+555.569337007" lastFinishedPulling="2026-05-28 17:52:36.26551841 +0000 UTC m=+557.369214106" observedRunningTime="2026-05-28 17:52:37.262264074 +0000 UTC m=+558.365959792" watchObservedRunningTime="2026-05-28 17:52:37.263290292 +0000 UTC m=+558.366986012"
May 28 17:52:37.461105 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:37.461080    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4l4g7"
May 28 17:52:37.506537 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:37.506441    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q64wc\" (UniqueName: \"kubernetes.io/projected/56bdf272-85b8-4330-be08-e1efafdc3852-kube-api-access-q64wc\") pod \"56bdf272-85b8-4330-be08-e1efafdc3852\" (UID: \"56bdf272-85b8-4330-be08-e1efafdc3852\") "
May 28 17:52:37.508749 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:37.508715    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56bdf272-85b8-4330-be08-e1efafdc3852-kube-api-access-q64wc" (OuterVolumeSpecName: "kube-api-access-q64wc") pod "56bdf272-85b8-4330-be08-e1efafdc3852" (UID: "56bdf272-85b8-4330-be08-e1efafdc3852"). InnerVolumeSpecName "kube-api-access-q64wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:52:37.607358 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:37.607328    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q64wc\" (UniqueName: \"kubernetes.io/projected/56bdf272-85b8-4330-be08-e1efafdc3852-kube-api-access-q64wc\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:52:38.231534 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:38.231500    2560 generic.go:358] "Generic (PLEG): container finished" podID="56bdf272-85b8-4330-be08-e1efafdc3852" containerID="9a19a401c3c099dd22955c7f8466e10bfdf239c745354dfe4826bea7799f42ef" exitCode=0
May 28 17:52:38.232066 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:38.231571    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4l4g7"
May 28 17:52:38.232066 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:38.231597    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4l4g7" event={"ID":"56bdf272-85b8-4330-be08-e1efafdc3852","Type":"ContainerDied","Data":"9a19a401c3c099dd22955c7f8466e10bfdf239c745354dfe4826bea7799f42ef"}
May 28 17:52:38.232066 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:38.231637    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4l4g7" event={"ID":"56bdf272-85b8-4330-be08-e1efafdc3852","Type":"ContainerDied","Data":"97f9bc0ad8310de9db14dcae5862394aefaadd77038b39c6e86dfbefa3338828"}
May 28 17:52:38.232066 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:38.231653    2560 scope.go:117] "RemoveContainer" containerID="9a19a401c3c099dd22955c7f8466e10bfdf239c745354dfe4826bea7799f42ef"
May 28 17:52:38.240089 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:38.240068    2560 scope.go:117] "RemoveContainer" containerID="9a19a401c3c099dd22955c7f8466e10bfdf239c745354dfe4826bea7799f42ef"
May 28 17:52:38.240352 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:52:38.240336    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a19a401c3c099dd22955c7f8466e10bfdf239c745354dfe4826bea7799f42ef\": container with ID starting with 9a19a401c3c099dd22955c7f8466e10bfdf239c745354dfe4826bea7799f42ef not found: ID does not exist" containerID="9a19a401c3c099dd22955c7f8466e10bfdf239c745354dfe4826bea7799f42ef"
May 28 17:52:38.240395 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:38.240363    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a19a401c3c099dd22955c7f8466e10bfdf239c745354dfe4826bea7799f42ef"} err="failed to get container status \"9a19a401c3c099dd22955c7f8466e10bfdf239c745354dfe4826bea7799f42ef\": rpc error: code = NotFound desc = could not find container \"9a19a401c3c099dd22955c7f8466e10bfdf239c745354dfe4826bea7799f42ef\": container with ID starting with 9a19a401c3c099dd22955c7f8466e10bfdf239c745354dfe4826bea7799f42ef not found: ID does not exist"
May 28 17:52:38.249620 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:38.249596    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4l4g7"]
May 28 17:52:38.253025 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:38.253004    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4l4g7"]
May 28 17:52:39.518826 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:39.518793    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56bdf272-85b8-4330-be08-e1efafdc3852" path="/var/lib/kubelet/pods/56bdf272-85b8-4330-be08-e1efafdc3852/volumes"
May 28 17:52:44.328946 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:44.328889    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-97fp7"
May 28 17:52:44.329442 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:44.328955    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-97fp7"
May 28 17:52:44.350831 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:44.350800    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-97fp7"
May 28 17:52:45.273611 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:52:45.273585    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-97fp7"
May 28 17:53:02.838561 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:02.838525    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-w5w9c"]
May 28 17:53:02.839085 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:02.838774    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56bdf272-85b8-4330-be08-e1efafdc3852" containerName="registry-server"
May 28 17:53:02.839085 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:02.838784    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="56bdf272-85b8-4330-be08-e1efafdc3852" containerName="registry-server"
May 28 17:53:02.839085 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:02.838849    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="56bdf272-85b8-4330-be08-e1efafdc3852" containerName="registry-server"
May 28 17:53:02.846938 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:02.846906    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-w5w9c"
May 28 17:53:02.850201 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:02.850173    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-5vclw\""
May 28 17:53:02.904021 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:02.903980    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-w5w9c"]
May 28 17:53:03.015818 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:03.015786    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qdwr\" (UniqueName: \"kubernetes.io/projected/d381a90c-4d7c-4823-9e12-28fbc9a90ab4-kube-api-access-2qdwr\") pod \"authorino-operator-657f44b778-w5w9c\" (UID: \"d381a90c-4d7c-4823-9e12-28fbc9a90ab4\") " pod="kuadrant-system/authorino-operator-657f44b778-w5w9c"
May 28 17:53:03.117107 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:03.117030    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2qdwr\" (UniqueName: \"kubernetes.io/projected/d381a90c-4d7c-4823-9e12-28fbc9a90ab4-kube-api-access-2qdwr\") pod \"authorino-operator-657f44b778-w5w9c\" (UID: \"d381a90c-4d7c-4823-9e12-28fbc9a90ab4\") " pod="kuadrant-system/authorino-operator-657f44b778-w5w9c"
May 28 17:53:03.143688 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:03.143661    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qdwr\" (UniqueName: \"kubernetes.io/projected/d381a90c-4d7c-4823-9e12-28fbc9a90ab4-kube-api-access-2qdwr\") pod \"authorino-operator-657f44b778-w5w9c\" (UID: \"d381a90c-4d7c-4823-9e12-28fbc9a90ab4\") " pod="kuadrant-system/authorino-operator-657f44b778-w5w9c"
May 28 17:53:03.156537 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:03.156517    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-w5w9c"
May 28 17:53:03.318987 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:03.318904    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-w5w9c"]
May 28 17:53:03.321681 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:53:03.321651    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd381a90c_4d7c_4823_9e12_28fbc9a90ab4.slice/crio-3b7b25e9158414cc45c111550d0530f649fa2f961429bdd53713fbbde2b38724 WatchSource:0}: Error finding container 3b7b25e9158414cc45c111550d0530f649fa2f961429bdd53713fbbde2b38724: Status 404 returned error can't find the container with id 3b7b25e9158414cc45c111550d0530f649fa2f961429bdd53713fbbde2b38724
May 28 17:53:04.310511 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:04.310463    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-w5w9c" event={"ID":"d381a90c-4d7c-4823-9e12-28fbc9a90ab4","Type":"ContainerStarted","Data":"3b7b25e9158414cc45c111550d0530f649fa2f961429bdd53713fbbde2b38724"}
May 28 17:53:06.317940 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:06.317882    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-w5w9c" event={"ID":"d381a90c-4d7c-4823-9e12-28fbc9a90ab4","Type":"ContainerStarted","Data":"66178c269abeec214fb06cde231249b63e5138666de53f64b94dfedec2e34361"}
May 28 17:53:06.318360 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:06.318013    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-w5w9c"
May 28 17:53:06.389102 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:06.389035    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-w5w9c" podStartSLOduration=1.880622287 podStartE2EDuration="4.389021193s" podCreationTimestamp="2026-05-28 17:53:02 +0000 UTC" firstStartedPulling="2026-05-28 17:53:03.323460721 +0000 UTC m=+584.427156417" lastFinishedPulling="2026-05-28 17:53:05.831859612 +0000 UTC m=+586.935555323" observedRunningTime="2026-05-28 17:53:06.388114084 +0000 UTC m=+587.491809794" watchObservedRunningTime="2026-05-28 17:53:06.389021193 +0000 UTC m=+587.492716911"
May 28 17:53:14.839246 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:14.839211    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"]
May 28 17:53:14.844098 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:14.844082    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"
May 28 17:53:14.848325 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:14.848293    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5ljqb\""
May 28 17:53:14.872513 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:14.872467    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"]
May 28 17:53:14.914081 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:14.914026    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pzh5\" (UniqueName: \"kubernetes.io/projected/331c4e4c-6582-488a-9d60-da26242e50d1-kube-api-access-6pzh5\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j4wpm\" (UID: \"331c4e4c-6582-488a-9d60-da26242e50d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"
May 28 17:53:14.914265 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:14.914109    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/331c4e4c-6582-488a-9d60-da26242e50d1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j4wpm\" (UID: \"331c4e4c-6582-488a-9d60-da26242e50d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"
May 28 17:53:15.015248 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:15.015211    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6pzh5\" (UniqueName: \"kubernetes.io/projected/331c4e4c-6582-488a-9d60-da26242e50d1-kube-api-access-6pzh5\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j4wpm\" (UID: \"331c4e4c-6582-488a-9d60-da26242e50d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"
May 28 17:53:15.015402 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:15.015260    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/331c4e4c-6582-488a-9d60-da26242e50d1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j4wpm\" (UID: \"331c4e4c-6582-488a-9d60-da26242e50d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"
May 28 17:53:15.015641 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:15.015625    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/331c4e4c-6582-488a-9d60-da26242e50d1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j4wpm\" (UID: \"331c4e4c-6582-488a-9d60-da26242e50d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"
May 28 17:53:15.027642 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:15.027610    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pzh5\" (UniqueName: \"kubernetes.io/projected/331c4e4c-6582-488a-9d60-da26242e50d1-kube-api-access-6pzh5\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-j4wpm\" (UID: \"331c4e4c-6582-488a-9d60-da26242e50d1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"
May 28 17:53:15.153466 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:15.153373    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"
May 28 17:53:15.320424 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:15.320350    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"]
May 28 17:53:15.323538 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:53:15.323505    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod331c4e4c_6582_488a_9d60_da26242e50d1.slice/crio-410f1ad09d047453a071618f40685268a191cc0d50140dbf321dfd0b8494296c WatchSource:0}: Error finding container 410f1ad09d047453a071618f40685268a191cc0d50140dbf321dfd0b8494296c: Status 404 returned error can't find the container with id 410f1ad09d047453a071618f40685268a191cc0d50140dbf321dfd0b8494296c
May 28 17:53:15.346101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:15.346067    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm" event={"ID":"331c4e4c-6582-488a-9d60-da26242e50d1","Type":"ContainerStarted","Data":"410f1ad09d047453a071618f40685268a191cc0d50140dbf321dfd0b8494296c"}
May 28 17:53:17.323349 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:17.323314    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-w5w9c"
May 28 17:53:20.365629 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:20.365588    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm" event={"ID":"331c4e4c-6582-488a-9d60-da26242e50d1","Type":"ContainerStarted","Data":"42ca1e65d876da1796f754e52b7cc5252a3006375142acc3927e070b180133e8"}
May 28 17:53:20.366133 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:20.365831    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"
May 28 17:53:20.487548 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:20.487499    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm" podStartSLOduration=1.948009754 podStartE2EDuration="6.487487099s" podCreationTimestamp="2026-05-28 17:53:14 +0000 UTC" firstStartedPulling="2026-05-28 17:53:15.32570867 +0000 UTC m=+596.429404366" lastFinishedPulling="2026-05-28 17:53:19.865186016 +0000 UTC m=+600.968881711" observedRunningTime="2026-05-28 17:53:20.482590098 +0000 UTC m=+601.586285810" watchObservedRunningTime="2026-05-28 17:53:20.487487099 +0000 UTC m=+601.591182818"
May 28 17:53:31.371239 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:31.371210    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"
May 28 17:53:33.402828 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.402789    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"]
May 28 17:53:33.403294 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.403041    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm" podUID="331c4e4c-6582-488a-9d60-da26242e50d1" containerName="manager" containerID="cri-o://42ca1e65d876da1796f754e52b7cc5252a3006375142acc3927e070b180133e8" gracePeriod=2
May 28 17:53:33.413816 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.413785    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"]
May 28 17:53:33.448693 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.448669    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"]
May 28 17:53:33.448978 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.448964    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="331c4e4c-6582-488a-9d60-da26242e50d1" containerName="manager"
May 28 17:53:33.449035 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.448982    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="331c4e4c-6582-488a-9d60-da26242e50d1" containerName="manager"
May 28 17:53:33.449073 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.449039    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="331c4e4c-6582-488a-9d60-da26242e50d1" containerName="manager"
May 28 17:53:33.451693 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.451667    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"
May 28 17:53:33.455354 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.455324    2560 status_manager.go:895] "Failed to get status for pod" podUID="331c4e4c-6582-488a-9d60-da26242e50d1" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-j4wpm\" is forbidden: User \"system:node:ip-10-0-135-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-41.ec2.internal' and this object"
May 28 17:53:33.477206 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.477181    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"]
May 28 17:53:33.558327 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.558293    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm2zk\" (UniqueName: \"kubernetes.io/projected/19115f55-5b6c-4c76-9506-feecdbd4ea93-kube-api-access-vm2zk\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-5gjxt\" (UID: \"19115f55-5b6c-4c76-9506-feecdbd4ea93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"
May 28 17:53:33.558491 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.558354    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/19115f55-5b6c-4c76-9506-feecdbd4ea93-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-5gjxt\" (UID: \"19115f55-5b6c-4c76-9506-feecdbd4ea93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"
May 28 17:53:33.635508 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.635476    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"
May 28 17:53:33.658958 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.658876    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/331c4e4c-6582-488a-9d60-da26242e50d1-extensions-socket-volume\") pod \"331c4e4c-6582-488a-9d60-da26242e50d1\" (UID: \"331c4e4c-6582-488a-9d60-da26242e50d1\") "
May 28 17:53:33.658958 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.658911    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pzh5\" (UniqueName: \"kubernetes.io/projected/331c4e4c-6582-488a-9d60-da26242e50d1-kube-api-access-6pzh5\") pod \"331c4e4c-6582-488a-9d60-da26242e50d1\" (UID: \"331c4e4c-6582-488a-9d60-da26242e50d1\") "
May 28 17:53:33.659138 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.659088    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vm2zk\" (UniqueName: \"kubernetes.io/projected/19115f55-5b6c-4c76-9506-feecdbd4ea93-kube-api-access-vm2zk\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-5gjxt\" (UID: \"19115f55-5b6c-4c76-9506-feecdbd4ea93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"
May 28 17:53:33.659196 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.659184    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/19115f55-5b6c-4c76-9506-feecdbd4ea93-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-5gjxt\" (UID: \"19115f55-5b6c-4c76-9506-feecdbd4ea93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"
May 28 17:53:33.659429 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.659402    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/331c4e4c-6582-488a-9d60-da26242e50d1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "331c4e4c-6582-488a-9d60-da26242e50d1" (UID: "331c4e4c-6582-488a-9d60-da26242e50d1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 17:53:33.659538 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.659518    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/19115f55-5b6c-4c76-9506-feecdbd4ea93-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-5gjxt\" (UID: \"19115f55-5b6c-4c76-9506-feecdbd4ea93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"
May 28 17:53:33.661124 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.661107    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/331c4e4c-6582-488a-9d60-da26242e50d1-kube-api-access-6pzh5" (OuterVolumeSpecName: "kube-api-access-6pzh5") pod "331c4e4c-6582-488a-9d60-da26242e50d1" (UID: "331c4e4c-6582-488a-9d60-da26242e50d1"). InnerVolumeSpecName "kube-api-access-6pzh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:53:33.672654 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.672635    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm2zk\" (UniqueName: \"kubernetes.io/projected/19115f55-5b6c-4c76-9506-feecdbd4ea93-kube-api-access-vm2zk\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-5gjxt\" (UID: \"19115f55-5b6c-4c76-9506-feecdbd4ea93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"
May 28 17:53:33.760505 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.760466    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/331c4e4c-6582-488a-9d60-da26242e50d1-extensions-socket-volume\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:53:33.760505 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.760497    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pzh5\" (UniqueName: \"kubernetes.io/projected/331c4e4c-6582-488a-9d60-da26242e50d1-kube-api-access-6pzh5\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:53:33.792039 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.791995    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"
May 28 17:53:33.971066 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:33.967007    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"]
May 28 17:53:34.408470 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:34.408421    2560 generic.go:358] "Generic (PLEG): container finished" podID="331c4e4c-6582-488a-9d60-da26242e50d1" containerID="42ca1e65d876da1796f754e52b7cc5252a3006375142acc3927e070b180133e8" exitCode=0
May 28 17:53:34.409006 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:34.408516    2560 scope.go:117] "RemoveContainer" containerID="42ca1e65d876da1796f754e52b7cc5252a3006375142acc3927e070b180133e8"
May 28 17:53:34.409006 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:34.408542    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-j4wpm"
May 28 17:53:34.411123 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:34.411000    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt" event={"ID":"19115f55-5b6c-4c76-9506-feecdbd4ea93","Type":"ContainerStarted","Data":"ae7ee857f8d68f35911023efc02b703917917f8f03ed2d36a91d34c5caf5dca7"}
May 28 17:53:34.411123 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:34.411045    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt" event={"ID":"19115f55-5b6c-4c76-9506-feecdbd4ea93","Type":"ContainerStarted","Data":"2d32cd82c68827877af90c9b95dbefd8c20fb4085bfe895a6bc046ae39ea10aa"}
May 28 17:53:34.411123 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:34.411082    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"
May 28 17:53:34.427149 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:34.427129    2560 scope.go:117] "RemoveContainer" containerID="42ca1e65d876da1796f754e52b7cc5252a3006375142acc3927e070b180133e8"
May 28 17:53:34.427443 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:53:34.427422    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42ca1e65d876da1796f754e52b7cc5252a3006375142acc3927e070b180133e8\": container with ID starting with 42ca1e65d876da1796f754e52b7cc5252a3006375142acc3927e070b180133e8 not found: ID does not exist" containerID="42ca1e65d876da1796f754e52b7cc5252a3006375142acc3927e070b180133e8"
May 28 17:53:34.427488 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:34.427455    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42ca1e65d876da1796f754e52b7cc5252a3006375142acc3927e070b180133e8"} err="failed to get container status \"42ca1e65d876da1796f754e52b7cc5252a3006375142acc3927e070b180133e8\": rpc error: code = NotFound desc = could not find container \"42ca1e65d876da1796f754e52b7cc5252a3006375142acc3927e070b180133e8\": container with ID starting with 42ca1e65d876da1796f754e52b7cc5252a3006375142acc3927e070b180133e8 not found: ID does not exist"
May 28 17:53:34.440563 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:34.440476    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt" podStartSLOduration=1.440459713 podStartE2EDuration="1.440459713s" podCreationTimestamp="2026-05-28 17:53:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:53:34.438366074 +0000 UTC m=+615.542061806" watchObservedRunningTime="2026-05-28 17:53:34.440459713 +0000 UTC m=+615.544155500"
May 28 17:53:35.517530 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:35.517492    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="331c4e4c-6582-488a-9d60-da26242e50d1" path="/var/lib/kubelet/pods/331c4e4c-6582-488a-9d60-da26242e50d1/volumes"
May 28 17:53:45.417137 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:45.417097    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"
May 28 17:53:51.317511 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:51.317467    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"]
May 28 17:53:51.317909 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:51.317731    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt" podUID="19115f55-5b6c-4c76-9506-feecdbd4ea93" containerName="manager" containerID="cri-o://ae7ee857f8d68f35911023efc02b703917917f8f03ed2d36a91d34c5caf5dca7" gracePeriod=10
May 28 17:53:51.466459 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:51.466431    2560 generic.go:358] "Generic (PLEG): container finished" podID="19115f55-5b6c-4c76-9506-feecdbd4ea93" containerID="ae7ee857f8d68f35911023efc02b703917917f8f03ed2d36a91d34c5caf5dca7" exitCode=0
May 28 17:53:51.466595 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:51.466485    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt" event={"ID":"19115f55-5b6c-4c76-9506-feecdbd4ea93","Type":"ContainerDied","Data":"ae7ee857f8d68f35911023efc02b703917917f8f03ed2d36a91d34c5caf5dca7"}
May 28 17:53:51.558963 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:51.558923    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"
May 28 17:53:51.595089 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:51.595029    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/19115f55-5b6c-4c76-9506-feecdbd4ea93-extensions-socket-volume\") pod \"19115f55-5b6c-4c76-9506-feecdbd4ea93\" (UID: \"19115f55-5b6c-4c76-9506-feecdbd4ea93\") "
May 28 17:53:51.595196 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:51.595116    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vm2zk\" (UniqueName: \"kubernetes.io/projected/19115f55-5b6c-4c76-9506-feecdbd4ea93-kube-api-access-vm2zk\") pod \"19115f55-5b6c-4c76-9506-feecdbd4ea93\" (UID: \"19115f55-5b6c-4c76-9506-feecdbd4ea93\") "
May 28 17:53:51.600183 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:51.596234    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19115f55-5b6c-4c76-9506-feecdbd4ea93-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "19115f55-5b6c-4c76-9506-feecdbd4ea93" (UID: "19115f55-5b6c-4c76-9506-feecdbd4ea93"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 17:53:51.601237 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:51.601207    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19115f55-5b6c-4c76-9506-feecdbd4ea93-kube-api-access-vm2zk" (OuterVolumeSpecName: "kube-api-access-vm2zk") pod "19115f55-5b6c-4c76-9506-feecdbd4ea93" (UID: "19115f55-5b6c-4c76-9506-feecdbd4ea93"). InnerVolumeSpecName "kube-api-access-vm2zk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:53:51.696627 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:51.696590    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vm2zk\" (UniqueName: \"kubernetes.io/projected/19115f55-5b6c-4c76-9506-feecdbd4ea93-kube-api-access-vm2zk\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:53:51.696627 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:51.696620    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/19115f55-5b6c-4c76-9506-feecdbd4ea93-extensions-socket-volume\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:53:52.470654 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:52.470617    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt" event={"ID":"19115f55-5b6c-4c76-9506-feecdbd4ea93","Type":"ContainerDied","Data":"2d32cd82c68827877af90c9b95dbefd8c20fb4085bfe895a6bc046ae39ea10aa"}
May 28 17:53:52.470654 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:52.470657    2560 scope.go:117] "RemoveContainer" containerID="ae7ee857f8d68f35911023efc02b703917917f8f03ed2d36a91d34c5caf5dca7"
May 28 17:53:52.471133 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:52.470666    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"
May 28 17:53:52.498910 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:52.498878    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"]
May 28 17:53:52.502989 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:52.502961    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-5gjxt"]
May 28 17:53:53.517869 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:53:53.517828    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19115f55-5b6c-4c76-9506-feecdbd4ea93" path="/var/lib/kubelet/pods/19115f55-5b6c-4c76-9506-feecdbd4ea93/volumes"
May 28 17:54:07.666350 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.666313    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"]
May 28 17:54:07.668748 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.666691    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19115f55-5b6c-4c76-9506-feecdbd4ea93" containerName="manager"
May 28 17:54:07.668748 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.666710    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="19115f55-5b6c-4c76-9506-feecdbd4ea93" containerName="manager"
May 28 17:54:07.668748 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.666777    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="19115f55-5b6c-4c76-9506-feecdbd4ea93" containerName="manager"
May 28 17:54:07.669692 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.669671    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.672172 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.672151    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-scsmz\""
May 28 17:54:07.688514 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.688480    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"]
May 28 17:54:07.823263 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.823222    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/214a7f13-c84e-42d1-b854-a8cc63b9de71-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.823467 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.823275    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/214a7f13-c84e-42d1-b854-a8cc63b9de71-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.823467 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.823301    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/214a7f13-c84e-42d1-b854-a8cc63b9de71-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.823467 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.823408    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/214a7f13-c84e-42d1-b854-a8cc63b9de71-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.823467 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.823462    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/214a7f13-c84e-42d1-b854-a8cc63b9de71-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.823629 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.823511    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/214a7f13-c84e-42d1-b854-a8cc63b9de71-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.823629 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.823534    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjz9v\" (UniqueName: \"kubernetes.io/projected/214a7f13-c84e-42d1-b854-a8cc63b9de71-kube-api-access-xjz9v\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.823629 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.823569    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/214a7f13-c84e-42d1-b854-a8cc63b9de71-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.823629 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.823608    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/214a7f13-c84e-42d1-b854-a8cc63b9de71-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.924946 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.924841    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/214a7f13-c84e-42d1-b854-a8cc63b9de71-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.924946 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.924888    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/214a7f13-c84e-42d1-b854-a8cc63b9de71-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.924946 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.924915    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/214a7f13-c84e-42d1-b854-a8cc63b9de71-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.925186 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.924964    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/214a7f13-c84e-42d1-b854-a8cc63b9de71-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.925186 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.924985    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/214a7f13-c84e-42d1-b854-a8cc63b9de71-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.925186 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.925017    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/214a7f13-c84e-42d1-b854-a8cc63b9de71-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.925186 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.925049    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/214a7f13-c84e-42d1-b854-a8cc63b9de71-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.925186 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.925094    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/214a7f13-c84e-42d1-b854-a8cc63b9de71-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.925186 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.925118    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xjz9v\" (UniqueName: \"kubernetes.io/projected/214a7f13-c84e-42d1-b854-a8cc63b9de71-kube-api-access-xjz9v\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.925457 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.925375    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/214a7f13-c84e-42d1-b854-a8cc63b9de71-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.925512 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.925495    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/214a7f13-c84e-42d1-b854-a8cc63b9de71-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.925575 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.925552    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/214a7f13-c84e-42d1-b854-a8cc63b9de71-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.925660 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.925641    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/214a7f13-c84e-42d1-b854-a8cc63b9de71-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.925867 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.925848    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/214a7f13-c84e-42d1-b854-a8cc63b9de71-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.927589 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.927569    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/214a7f13-c84e-42d1-b854-a8cc63b9de71-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.927814 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.927795    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/214a7f13-c84e-42d1-b854-a8cc63b9de71-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.932907 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.932887    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjz9v\" (UniqueName: \"kubernetes.io/projected/214a7f13-c84e-42d1-b854-a8cc63b9de71-kube-api-access-xjz9v\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.932907 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.932897    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/214a7f13-c84e-42d1-b854-a8cc63b9de71-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-7tpzj\" (UID: \"214a7f13-c84e-42d1-b854-a8cc63b9de71\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:07.990063 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:07.990032    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:08.122821 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:08.122767    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"]
May 28 17:54:08.125374 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:54:08.125346    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod214a7f13_c84e_42d1_b854_a8cc63b9de71.slice/crio-37fb28107156a73c36b413495d2cdc1fa85130c940ec102328519d013d1736e7 WatchSource:0}: Error finding container 37fb28107156a73c36b413495d2cdc1fa85130c940ec102328519d013d1736e7: Status 404 returned error can't find the container with id 37fb28107156a73c36b413495d2cdc1fa85130c940ec102328519d013d1736e7
May 28 17:54:08.127350 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:08.127315    2560 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
May 28 17:54:08.127445 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:08.127379    2560 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
May 28 17:54:08.127445 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:08.127406    2560 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
May 28 17:54:08.519836 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:08.519750    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj" event={"ID":"214a7f13-c84e-42d1-b854-a8cc63b9de71","Type":"ContainerStarted","Data":"570b9e8cd06bd3117cbaa2a2a1753b29e30a399261ae36ef6ff557c86e39ba5b"}
May 28 17:54:08.519836 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:08.519785    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj" event={"ID":"214a7f13-c84e-42d1-b854-a8cc63b9de71","Type":"ContainerStarted","Data":"37fb28107156a73c36b413495d2cdc1fa85130c940ec102328519d013d1736e7"}
May 28 17:54:08.538464 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:08.538404    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj" podStartSLOduration=1.538386854 podStartE2EDuration="1.538386854s" podCreationTimestamp="2026-05-28 17:54:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:54:08.537077058 +0000 UTC m=+649.640772776" watchObservedRunningTime="2026-05-28 17:54:08.538386854 +0000 UTC m=+649.642082573"
May 28 17:54:08.990771 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:08.990725    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:09.995149 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:09.995115    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:10.525573 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:10.525533    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:10.526657 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:10.526635    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-7tpzj"
May 28 17:54:11.805869 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:11.805821    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-dmpg9"]
May 28 17:54:11.808939 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:11.808913    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9"
May 28 17:54:11.812134 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:11.812106    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cfxjz\""
May 28 17:54:11.812134 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:11.812107    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
May 28 17:54:11.833831 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:11.833798    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-dmpg9"]
May 28 17:54:11.920433 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:11.920385    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-dmpg9"]
May 28 17:54:11.961101 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:11.961069    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgv6f\" (UniqueName: \"kubernetes.io/projected/9d60efaf-f15d-4b4f-b2b3-5d9655232522-kube-api-access-sgv6f\") pod \"limitador-limitador-7d549b5b-dmpg9\" (UID: \"9d60efaf-f15d-4b4f-b2b3-5d9655232522\") " pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9"
May 28 17:54:11.961297 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:11.961116    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d60efaf-f15d-4b4f-b2b3-5d9655232522-config-file\") pod \"limitador-limitador-7d549b5b-dmpg9\" (UID: \"9d60efaf-f15d-4b4f-b2b3-5d9655232522\") " pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9"
May 28 17:54:12.061977 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:12.061854    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sgv6f\" (UniqueName: \"kubernetes.io/projected/9d60efaf-f15d-4b4f-b2b3-5d9655232522-kube-api-access-sgv6f\") pod \"limitador-limitador-7d549b5b-dmpg9\" (UID: \"9d60efaf-f15d-4b4f-b2b3-5d9655232522\") " pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9"
May 28 17:54:12.061977 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:12.061956    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d60efaf-f15d-4b4f-b2b3-5d9655232522-config-file\") pod \"limitador-limitador-7d549b5b-dmpg9\" (UID: \"9d60efaf-f15d-4b4f-b2b3-5d9655232522\") " pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9"
May 28 17:54:12.062643 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:12.062619    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d60efaf-f15d-4b4f-b2b3-5d9655232522-config-file\") pod \"limitador-limitador-7d549b5b-dmpg9\" (UID: \"9d60efaf-f15d-4b4f-b2b3-5d9655232522\") " pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9"
May 28 17:54:12.072567 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:12.072537    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgv6f\" (UniqueName: \"kubernetes.io/projected/9d60efaf-f15d-4b4f-b2b3-5d9655232522-kube-api-access-sgv6f\") pod \"limitador-limitador-7d549b5b-dmpg9\" (UID: \"9d60efaf-f15d-4b4f-b2b3-5d9655232522\") " pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9"
May 28 17:54:12.119652 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:12.119614    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9"
May 28 17:54:12.251712 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:12.251624    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-dmpg9"]
May 28 17:54:12.254300 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:54:12.254266    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d60efaf_f15d_4b4f_b2b3_5d9655232522.slice/crio-7d0cbd14c703e14b0c4d1bb768b7f812e586c3013c167d4b6b8f53ec3eba40cb WatchSource:0}: Error finding container 7d0cbd14c703e14b0c4d1bb768b7f812e586c3013c167d4b6b8f53ec3eba40cb: Status 404 returned error can't find the container with id 7d0cbd14c703e14b0c4d1bb768b7f812e586c3013c167d4b6b8f53ec3eba40cb
May 28 17:54:12.533462 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:12.533424    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9" event={"ID":"9d60efaf-f15d-4b4f-b2b3-5d9655232522","Type":"ContainerStarted","Data":"7d0cbd14c703e14b0c4d1bb768b7f812e586c3013c167d4b6b8f53ec3eba40cb"}
May 28 17:54:12.751494 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:12.751460    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-l57h8"]
May 28 17:54:12.754475 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:12.754459    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-l57h8"
May 28 17:54:12.756849 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:12.756827    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lxz22\""
May 28 17:54:12.759393 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:12.759369    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-l57h8"]
May 28 17:54:12.868308 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:12.868243    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn9jg\" (UniqueName: \"kubernetes.io/projected/d58f73f8-d663-4ec6-b441-1f19834c37a9-kube-api-access-bn9jg\") pod \"authorino-7498df8756-l57h8\" (UID: \"d58f73f8-d663-4ec6-b441-1f19834c37a9\") " pod="kuadrant-system/authorino-7498df8756-l57h8"
May 28 17:54:12.969821 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:12.969779    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bn9jg\" (UniqueName: \"kubernetes.io/projected/d58f73f8-d663-4ec6-b441-1f19834c37a9-kube-api-access-bn9jg\") pod \"authorino-7498df8756-l57h8\" (UID: \"d58f73f8-d663-4ec6-b441-1f19834c37a9\") " pod="kuadrant-system/authorino-7498df8756-l57h8"
May 28 17:54:12.984197 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:12.984136    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn9jg\" (UniqueName: \"kubernetes.io/projected/d58f73f8-d663-4ec6-b441-1f19834c37a9-kube-api-access-bn9jg\") pod \"authorino-7498df8756-l57h8\" (UID: \"d58f73f8-d663-4ec6-b441-1f19834c37a9\") " pod="kuadrant-system/authorino-7498df8756-l57h8"
May 28 17:54:13.064973 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:13.064801    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-l57h8"
May 28 17:54:13.239222 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:13.239161    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-l57h8"]
May 28 17:54:13.242824 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:54:13.242774    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd58f73f8_d663_4ec6_b441_1f19834c37a9.slice/crio-873a6d100f94afcdee1a3151dcb17963c7311490248c9b2e5201b77af4c200b3 WatchSource:0}: Error finding container 873a6d100f94afcdee1a3151dcb17963c7311490248c9b2e5201b77af4c200b3: Status 404 returned error can't find the container with id 873a6d100f94afcdee1a3151dcb17963c7311490248c9b2e5201b77af4c200b3
May 28 17:54:13.538366 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:13.538329    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-l57h8" event={"ID":"d58f73f8-d663-4ec6-b441-1f19834c37a9","Type":"ContainerStarted","Data":"873a6d100f94afcdee1a3151dcb17963c7311490248c9b2e5201b77af4c200b3"}
May 28 17:54:17.560675 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:17.560632    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-l57h8" event={"ID":"d58f73f8-d663-4ec6-b441-1f19834c37a9","Type":"ContainerStarted","Data":"450228fa17f07bfb52329c070a946a60f6efdab65b22f5f7133521f32ba9e812"}
May 28 17:54:17.562044 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:17.562011    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9" event={"ID":"9d60efaf-f15d-4b4f-b2b3-5d9655232522","Type":"ContainerStarted","Data":"08899eaab20d108cf588c52e397df73be5390bd1bc05cbf8d41e25890030311c"}
May 28 17:54:17.562172 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:17.562129    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9"
May 28 17:54:17.576396 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:17.576347    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-l57h8" podStartSLOduration=1.9690264910000002 podStartE2EDuration="5.576333056s" podCreationTimestamp="2026-05-28 17:54:12 +0000 UTC" firstStartedPulling="2026-05-28 17:54:13.244708955 +0000 UTC m=+654.348404665" lastFinishedPulling="2026-05-28 17:54:16.852015522 +0000 UTC m=+657.955711230" observedRunningTime="2026-05-28 17:54:17.574305815 +0000 UTC m=+658.678001532" watchObservedRunningTime="2026-05-28 17:54:17.576333056 +0000 UTC m=+658.680028775"
May 28 17:54:17.590759 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:17.590709    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9" podStartSLOduration=2.001279761 podStartE2EDuration="6.590694985s" podCreationTimestamp="2026-05-28 17:54:11 +0000 UTC" firstStartedPulling="2026-05-28 17:54:12.256216825 +0000 UTC m=+653.359912521" lastFinishedPulling="2026-05-28 17:54:16.845632042 +0000 UTC m=+657.949327745" observedRunningTime="2026-05-28 17:54:17.589333532 +0000 UTC m=+658.693029250" watchObservedRunningTime="2026-05-28 17:54:17.590694985 +0000 UTC m=+658.694390761"
May 28 17:54:28.204511 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:28.204482    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-dmpg9"]
May 28 17:54:28.204975 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:28.204762    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9" podUID="9d60efaf-f15d-4b4f-b2b3-5d9655232522" containerName="limitador" containerID="cri-o://08899eaab20d108cf588c52e397df73be5390bd1bc05cbf8d41e25890030311c" gracePeriod=30
May 28 17:54:28.205466 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:28.205398    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9"
May 28 17:54:28.596680 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:28.596643    2560 generic.go:358] "Generic (PLEG): container finished" podID="9d60efaf-f15d-4b4f-b2b3-5d9655232522" containerID="08899eaab20d108cf588c52e397df73be5390bd1bc05cbf8d41e25890030311c" exitCode=0
May 28 17:54:28.596855 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:28.596718    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9" event={"ID":"9d60efaf-f15d-4b4f-b2b3-5d9655232522","Type":"ContainerDied","Data":"08899eaab20d108cf588c52e397df73be5390bd1bc05cbf8d41e25890030311c"}
May 28 17:54:29.156611 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:29.156591    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9"
May 28 17:54:29.306787 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:29.306755    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgv6f\" (UniqueName: \"kubernetes.io/projected/9d60efaf-f15d-4b4f-b2b3-5d9655232522-kube-api-access-sgv6f\") pod \"9d60efaf-f15d-4b4f-b2b3-5d9655232522\" (UID: \"9d60efaf-f15d-4b4f-b2b3-5d9655232522\") "
May 28 17:54:29.307279 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:29.306807    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d60efaf-f15d-4b4f-b2b3-5d9655232522-config-file\") pod \"9d60efaf-f15d-4b4f-b2b3-5d9655232522\" (UID: \"9d60efaf-f15d-4b4f-b2b3-5d9655232522\") "
May 28 17:54:29.307279 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:29.307198    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d60efaf-f15d-4b4f-b2b3-5d9655232522-config-file" (OuterVolumeSpecName: "config-file") pod "9d60efaf-f15d-4b4f-b2b3-5d9655232522" (UID: "9d60efaf-f15d-4b4f-b2b3-5d9655232522"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 17:54:29.308972 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:29.308952    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d60efaf-f15d-4b4f-b2b3-5d9655232522-kube-api-access-sgv6f" (OuterVolumeSpecName: "kube-api-access-sgv6f") pod "9d60efaf-f15d-4b4f-b2b3-5d9655232522" (UID: "9d60efaf-f15d-4b4f-b2b3-5d9655232522"). InnerVolumeSpecName "kube-api-access-sgv6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:54:29.407605 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:29.407567    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgv6f\" (UniqueName: \"kubernetes.io/projected/9d60efaf-f15d-4b4f-b2b3-5d9655232522-kube-api-access-sgv6f\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:54:29.407605 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:29.407599    2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d60efaf-f15d-4b4f-b2b3-5d9655232522-config-file\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:54:29.600470 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:29.600383    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9" event={"ID":"9d60efaf-f15d-4b4f-b2b3-5d9655232522","Type":"ContainerDied","Data":"7d0cbd14c703e14b0c4d1bb768b7f812e586c3013c167d4b6b8f53ec3eba40cb"}
May 28 17:54:29.600470 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:29.600425    2560 scope.go:117] "RemoveContainer" containerID="08899eaab20d108cf588c52e397df73be5390bd1bc05cbf8d41e25890030311c"
May 28 17:54:29.600470 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:29.600444    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-dmpg9"
May 28 17:54:29.616624 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:29.616601    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-dmpg9"]
May 28 17:54:29.618429 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:29.618407    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-dmpg9"]
May 28 17:54:31.518284 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:31.518251    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d60efaf-f15d-4b4f-b2b3-5d9655232522" path="/var/lib/kubelet/pods/9d60efaf-f15d-4b4f-b2b3-5d9655232522/volumes"
May 28 17:54:33.153768 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.153737    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-k2299"]
May 28 17:54:33.154172 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.154070    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d60efaf-f15d-4b4f-b2b3-5d9655232522" containerName="limitador"
May 28 17:54:33.154172 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.154083    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d60efaf-f15d-4b4f-b2b3-5d9655232522" containerName="limitador"
May 28 17:54:33.154172 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.154146    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="9d60efaf-f15d-4b4f-b2b3-5d9655232522" containerName="limitador"
May 28 17:54:33.158251 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.158228    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-k2299"
May 28 17:54:33.160568 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.160541    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qmcdw\""
May 28 17:54:33.160706 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.160568    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
May 28 17:54:33.164376 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.164351    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-k2299"]
May 28 17:54:33.239098 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.239061    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtl5z\" (UniqueName: \"kubernetes.io/projected/e44f759f-b13e-4295-b282-658fff395205-kube-api-access-vtl5z\") pod \"postgres-68c4fbbd6f-k2299\" (UID: \"e44f759f-b13e-4295-b282-658fff395205\") " pod="opendatahub/postgres-68c4fbbd6f-k2299"
May 28 17:54:33.239289 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.239104    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e44f759f-b13e-4295-b282-658fff395205-data\") pod \"postgres-68c4fbbd6f-k2299\" (UID: \"e44f759f-b13e-4295-b282-658fff395205\") " pod="opendatahub/postgres-68c4fbbd6f-k2299"
May 28 17:54:33.339638 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.339593    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vtl5z\" (UniqueName: \"kubernetes.io/projected/e44f759f-b13e-4295-b282-658fff395205-kube-api-access-vtl5z\") pod \"postgres-68c4fbbd6f-k2299\" (UID: \"e44f759f-b13e-4295-b282-658fff395205\") " pod="opendatahub/postgres-68c4fbbd6f-k2299"
May 28 17:54:33.339638 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.339643    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e44f759f-b13e-4295-b282-658fff395205-data\") pod \"postgres-68c4fbbd6f-k2299\" (UID: \"e44f759f-b13e-4295-b282-658fff395205\") " pod="opendatahub/postgres-68c4fbbd6f-k2299"
May 28 17:54:33.339989 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.339975    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e44f759f-b13e-4295-b282-658fff395205-data\") pod \"postgres-68c4fbbd6f-k2299\" (UID: \"e44f759f-b13e-4295-b282-658fff395205\") " pod="opendatahub/postgres-68c4fbbd6f-k2299"
May 28 17:54:33.347528 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.347496    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtl5z\" (UniqueName: \"kubernetes.io/projected/e44f759f-b13e-4295-b282-658fff395205-kube-api-access-vtl5z\") pod \"postgres-68c4fbbd6f-k2299\" (UID: \"e44f759f-b13e-4295-b282-658fff395205\") " pod="opendatahub/postgres-68c4fbbd6f-k2299"
May 28 17:54:33.472623 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.472555    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-k2299"
May 28 17:54:33.597670 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.597619    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-k2299"]
May 28 17:54:33.600139 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:54:33.600112    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode44f759f_b13e_4295_b282_658fff395205.slice/crio-a5a7b1a43a5548456440af948e6ae88ac24e3355ea08d81fdaf203a7766f0c5f WatchSource:0}: Error finding container a5a7b1a43a5548456440af948e6ae88ac24e3355ea08d81fdaf203a7766f0c5f: Status 404 returned error can't find the container with id a5a7b1a43a5548456440af948e6ae88ac24e3355ea08d81fdaf203a7766f0c5f
May 28 17:54:33.614597 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:33.614566    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-k2299" event={"ID":"e44f759f-b13e-4295-b282-658fff395205","Type":"ContainerStarted","Data":"a5a7b1a43a5548456440af948e6ae88ac24e3355ea08d81fdaf203a7766f0c5f"}
May 28 17:54:38.634213 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:38.634180    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-k2299" event={"ID":"e44f759f-b13e-4295-b282-658fff395205","Type":"ContainerStarted","Data":"9d44130d72aab188f10fa3bd5c441eae569b73d4044577c2b9054bfdb953428f"}
May 28 17:54:38.634638 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:38.634280    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-k2299"
May 28 17:54:38.649133 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:38.649086    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-k2299" podStartSLOduration=0.757650558 podStartE2EDuration="5.649070727s" podCreationTimestamp="2026-05-28 17:54:33 +0000 UTC" firstStartedPulling="2026-05-28 17:54:33.601704076 +0000 UTC m=+674.705399772" lastFinishedPulling="2026-05-28 17:54:38.493124246 +0000 UTC m=+679.596819941" observedRunningTime="2026-05-28 17:54:38.648153893 +0000 UTC m=+679.751849612" watchObservedRunningTime="2026-05-28 17:54:38.649070727 +0000 UTC m=+679.752766445"
May 28 17:54:44.668103 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:44.668073    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-k2299"
May 28 17:54:45.228034 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.227994    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-ql9dj"]
May 28 17:54:45.236094 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.236059    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-ql9dj"
May 28 17:54:45.238178 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.238151    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-ql9dj"]
May 28 17:54:45.335605 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.335558    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5knls\" (UniqueName: \"kubernetes.io/projected/fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779-kube-api-access-5knls\") pod \"authorino-8b475cf9f-ql9dj\" (UID: \"fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779\") " pod="kuadrant-system/authorino-8b475cf9f-ql9dj"
May 28 17:54:45.436950 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.436898    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5knls\" (UniqueName: \"kubernetes.io/projected/fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779-kube-api-access-5knls\") pod \"authorino-8b475cf9f-ql9dj\" (UID: \"fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779\") " pod="kuadrant-system/authorino-8b475cf9f-ql9dj"
May 28 17:54:45.445050 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.445019    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5knls\" (UniqueName: \"kubernetes.io/projected/fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779-kube-api-access-5knls\") pod \"authorino-8b475cf9f-ql9dj\" (UID: \"fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779\") " pod="kuadrant-system/authorino-8b475cf9f-ql9dj"
May 28 17:54:45.487127 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.487041    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-ql9dj"]
May 28 17:54:45.487333 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.487315    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-ql9dj"
May 28 17:54:45.511395 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.511359    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c889456b9-cbsxc"]
May 28 17:54:45.547946 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.547890    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c889456b9-cbsxc"]
May 28 17:54:45.548703 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.548615    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c889456b9-cbsxc"
May 28 17:54:45.625846 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.625821    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-ql9dj"]
May 28 17:54:45.628220 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:54:45.628195    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa5bb4b7_0b76_4f1c_8f4e_5ac09ac66779.slice/crio-3ab7930411e442242be84e2e1d26b01f3feeebb9ed8f57114f3130f41c9a822d WatchSource:0}: Error finding container 3ab7930411e442242be84e2e1d26b01f3feeebb9ed8f57114f3130f41c9a822d: Status 404 returned error can't find the container with id 3ab7930411e442242be84e2e1d26b01f3feeebb9ed8f57114f3130f41c9a822d
May 28 17:54:45.639055 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.639030    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbfh7\" (UniqueName: \"kubernetes.io/projected/a8f443a0-3fd4-43bd-ad7c-62170c213f11-kube-api-access-zbfh7\") pod \"authorino-c889456b9-cbsxc\" (UID: \"a8f443a0-3fd4-43bd-ad7c-62170c213f11\") " pod="kuadrant-system/authorino-c889456b9-cbsxc"
May 28 17:54:45.658884 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.658852    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-ql9dj" event={"ID":"fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779","Type":"ContainerStarted","Data":"3ab7930411e442242be84e2e1d26b01f3feeebb9ed8f57114f3130f41c9a822d"}
May 28 17:54:45.738582 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.738507    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c889456b9-cbsxc"]
May 28 17:54:45.738923 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:54:45.738729    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-zbfh7], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-c889456b9-cbsxc" podUID="a8f443a0-3fd4-43bd-ad7c-62170c213f11"
May 28 17:54:45.740252 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.740231    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zbfh7\" (UniqueName: \"kubernetes.io/projected/a8f443a0-3fd4-43bd-ad7c-62170c213f11-kube-api-access-zbfh7\") pod \"authorino-c889456b9-cbsxc\" (UID: \"a8f443a0-3fd4-43bd-ad7c-62170c213f11\") " pod="kuadrant-system/authorino-c889456b9-cbsxc"
May 28 17:54:45.757138 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.757105    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbfh7\" (UniqueName: \"kubernetes.io/projected/a8f443a0-3fd4-43bd-ad7c-62170c213f11-kube-api-access-zbfh7\") pod \"authorino-c889456b9-cbsxc\" (UID: \"a8f443a0-3fd4-43bd-ad7c-62170c213f11\") " pod="kuadrant-system/authorino-c889456b9-cbsxc"
May 28 17:54:45.767655 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.767625    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d8b7d4fd5-226bn"]
May 28 17:54:45.801841 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.801809    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d8b7d4fd5-226bn"]
May 28 17:54:45.802017 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.801952    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8b7d4fd5-226bn"
May 28 17:54:45.804446 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.804421    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
May 28 17:54:45.841258 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.841218    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ea459310-6743-487c-987e-efc530bd283f-tls-cert\") pod \"authorino-d8b7d4fd5-226bn\" (UID: \"ea459310-6743-487c-987e-efc530bd283f\") " pod="kuadrant-system/authorino-d8b7d4fd5-226bn"
May 28 17:54:45.841418 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.841289    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxtmc\" (UniqueName: \"kubernetes.io/projected/ea459310-6743-487c-987e-efc530bd283f-kube-api-access-fxtmc\") pod \"authorino-d8b7d4fd5-226bn\" (UID: \"ea459310-6743-487c-987e-efc530bd283f\") " pod="kuadrant-system/authorino-d8b7d4fd5-226bn"
May 28 17:54:45.942468 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.942428    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ea459310-6743-487c-987e-efc530bd283f-tls-cert\") pod \"authorino-d8b7d4fd5-226bn\" (UID: \"ea459310-6743-487c-987e-efc530bd283f\") " pod="kuadrant-system/authorino-d8b7d4fd5-226bn"
May 28 17:54:45.942630 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.942508    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fxtmc\" (UniqueName: \"kubernetes.io/projected/ea459310-6743-487c-987e-efc530bd283f-kube-api-access-fxtmc\") pod \"authorino-d8b7d4fd5-226bn\" (UID: \"ea459310-6743-487c-987e-efc530bd283f\") " pod="kuadrant-system/authorino-d8b7d4fd5-226bn"
May 28 17:54:45.945048 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.945018    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ea459310-6743-487c-987e-efc530bd283f-tls-cert\") pod \"authorino-d8b7d4fd5-226bn\" (UID: \"ea459310-6743-487c-987e-efc530bd283f\") " pod="kuadrant-system/authorino-d8b7d4fd5-226bn"
May 28 17:54:45.950228 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:45.950207    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxtmc\" (UniqueName: \"kubernetes.io/projected/ea459310-6743-487c-987e-efc530bd283f-kube-api-access-fxtmc\") pod \"authorino-d8b7d4fd5-226bn\" (UID: \"ea459310-6743-487c-987e-efc530bd283f\") " pod="kuadrant-system/authorino-d8b7d4fd5-226bn"
May 28 17:54:46.111472 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:46.111435    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8b7d4fd5-226bn"
May 28 17:54:46.269634 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:46.269533    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d8b7d4fd5-226bn"]
May 28 17:54:46.272249 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:54:46.272215    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea459310_6743_487c_987e_efc530bd283f.slice/crio-347abc0f9f50eb610b34316971d7a143f68b68409abbba7abf9f73bee335df43 WatchSource:0}: Error finding container 347abc0f9f50eb610b34316971d7a143f68b68409abbba7abf9f73bee335df43: Status 404 returned error can't find the container with id 347abc0f9f50eb610b34316971d7a143f68b68409abbba7abf9f73bee335df43
May 28 17:54:46.663236 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:46.663199    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8b7d4fd5-226bn" event={"ID":"ea459310-6743-487c-987e-efc530bd283f","Type":"ContainerStarted","Data":"347abc0f9f50eb610b34316971d7a143f68b68409abbba7abf9f73bee335df43"}
May 28 17:54:46.664575 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:46.664544    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-ql9dj" event={"ID":"fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779","Type":"ContainerStarted","Data":"0e2a17b8e9f0577ffa094db6006e6a0f522d1d82484953bef43bcf7ad8dd095f"}
May 28 17:54:46.664575 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:46.664552    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c889456b9-cbsxc"
May 28 17:54:46.664738 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:46.664572    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-8b475cf9f-ql9dj" podUID="fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779" containerName="authorino" containerID="cri-o://0e2a17b8e9f0577ffa094db6006e6a0f522d1d82484953bef43bcf7ad8dd095f" gracePeriod=30
May 28 17:54:46.670447 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:46.670427    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c889456b9-cbsxc"
May 28 17:54:46.680065 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:46.680021    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-8b475cf9f-ql9dj" podStartSLOduration=1.166223638 podStartE2EDuration="1.680007151s" podCreationTimestamp="2026-05-28 17:54:45 +0000 UTC" firstStartedPulling="2026-05-28 17:54:45.629879177 +0000 UTC m=+686.733574876" lastFinishedPulling="2026-05-28 17:54:46.143662682 +0000 UTC m=+687.247358389" observedRunningTime="2026-05-28 17:54:46.679378169 +0000 UTC m=+687.783073887" watchObservedRunningTime="2026-05-28 17:54:46.680007151 +0000 UTC m=+687.783702868"
May 28 17:54:46.750658 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:46.750621    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbfh7\" (UniqueName: \"kubernetes.io/projected/a8f443a0-3fd4-43bd-ad7c-62170c213f11-kube-api-access-zbfh7\") pod \"a8f443a0-3fd4-43bd-ad7c-62170c213f11\" (UID: \"a8f443a0-3fd4-43bd-ad7c-62170c213f11\") "
May 28 17:54:46.752860 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:46.752834    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8f443a0-3fd4-43bd-ad7c-62170c213f11-kube-api-access-zbfh7" (OuterVolumeSpecName: "kube-api-access-zbfh7") pod "a8f443a0-3fd4-43bd-ad7c-62170c213f11" (UID: "a8f443a0-3fd4-43bd-ad7c-62170c213f11"). InnerVolumeSpecName "kube-api-access-zbfh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:54:46.851377 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:46.851347    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbfh7\" (UniqueName: \"kubernetes.io/projected/a8f443a0-3fd4-43bd-ad7c-62170c213f11-kube-api-access-zbfh7\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:54:46.920457 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:46.920430    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-ql9dj"
May 28 17:54:46.952449 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:46.952407    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5knls\" (UniqueName: \"kubernetes.io/projected/fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779-kube-api-access-5knls\") pod \"fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779\" (UID: \"fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779\") "
May 28 17:54:46.954963 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:46.954691    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779-kube-api-access-5knls" (OuterVolumeSpecName: "kube-api-access-5knls") pod "fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779" (UID: "fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779"). InnerVolumeSpecName "kube-api-access-5knls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:54:47.053105 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.053074    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5knls\" (UniqueName: \"kubernetes.io/projected/fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779-kube-api-access-5knls\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:54:47.668745 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.668715    2560 generic.go:358] "Generic (PLEG): container finished" podID="fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779" containerID="0e2a17b8e9f0577ffa094db6006e6a0f522d1d82484953bef43bcf7ad8dd095f" exitCode=0
May 28 17:54:47.668951 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.668765    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-ql9dj"
May 28 17:54:47.668951 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.668797    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-ql9dj" event={"ID":"fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779","Type":"ContainerDied","Data":"0e2a17b8e9f0577ffa094db6006e6a0f522d1d82484953bef43bcf7ad8dd095f"}
May 28 17:54:47.668951 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.668837    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-ql9dj" event={"ID":"fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779","Type":"ContainerDied","Data":"3ab7930411e442242be84e2e1d26b01f3feeebb9ed8f57114f3130f41c9a822d"}
May 28 17:54:47.668951 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.668858    2560 scope.go:117] "RemoveContainer" containerID="0e2a17b8e9f0577ffa094db6006e6a0f522d1d82484953bef43bcf7ad8dd095f"
May 28 17:54:47.670397 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.670332    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c889456b9-cbsxc"
May 28 17:54:47.670397 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.670367    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8b7d4fd5-226bn" event={"ID":"ea459310-6743-487c-987e-efc530bd283f","Type":"ContainerStarted","Data":"17edc499e5bec8682b9f561ca20fc6e397037910763bc1c857b7de045bc445d9"}
May 28 17:54:47.677102 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.677086    2560 scope.go:117] "RemoveContainer" containerID="0e2a17b8e9f0577ffa094db6006e6a0f522d1d82484953bef43bcf7ad8dd095f"
May 28 17:54:47.677379 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:54:47.677362    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e2a17b8e9f0577ffa094db6006e6a0f522d1d82484953bef43bcf7ad8dd095f\": container with ID starting with 0e2a17b8e9f0577ffa094db6006e6a0f522d1d82484953bef43bcf7ad8dd095f not found: ID does not exist" containerID="0e2a17b8e9f0577ffa094db6006e6a0f522d1d82484953bef43bcf7ad8dd095f"
May 28 17:54:47.677428 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.677388    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e2a17b8e9f0577ffa094db6006e6a0f522d1d82484953bef43bcf7ad8dd095f"} err="failed to get container status \"0e2a17b8e9f0577ffa094db6006e6a0f522d1d82484953bef43bcf7ad8dd095f\": rpc error: code = NotFound desc = could not find container \"0e2a17b8e9f0577ffa094db6006e6a0f522d1d82484953bef43bcf7ad8dd095f\": container with ID starting with 0e2a17b8e9f0577ffa094db6006e6a0f522d1d82484953bef43bcf7ad8dd095f not found: ID does not exist"
May 28 17:54:47.687711 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.687683    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-ql9dj"]
May 28 17:54:47.691311 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.691287    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-ql9dj"]
May 28 17:54:47.711129 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.711034    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-d8b7d4fd5-226bn" podStartSLOduration=2.221023578 podStartE2EDuration="2.71100705s" podCreationTimestamp="2026-05-28 17:54:45 +0000 UTC" firstStartedPulling="2026-05-28 17:54:46.273654393 +0000 UTC m=+687.377350089" lastFinishedPulling="2026-05-28 17:54:46.763637862 +0000 UTC m=+687.867333561" observedRunningTime="2026-05-28 17:54:47.710624649 +0000 UTC m=+688.814320369" watchObservedRunningTime="2026-05-28 17:54:47.71100705 +0000 UTC m=+688.814702770"
May 28 17:54:47.732087 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.732048    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c889456b9-cbsxc"]
May 28 17:54:47.737599 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.737572    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-l57h8"]
May 28 17:54:47.737801 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.737779    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-l57h8" podUID="d58f73f8-d663-4ec6-b441-1f19834c37a9" containerName="authorino" containerID="cri-o://450228fa17f07bfb52329c070a946a60f6efdab65b22f5f7133521f32ba9e812" gracePeriod=30
May 28 17:54:47.740090 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.740063    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c889456b9-cbsxc"]
May 28 17:54:47.996285 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:47.996260    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-l57h8"
May 28 17:54:48.061205 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:48.061166    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn9jg\" (UniqueName: \"kubernetes.io/projected/d58f73f8-d663-4ec6-b441-1f19834c37a9-kube-api-access-bn9jg\") pod \"d58f73f8-d663-4ec6-b441-1f19834c37a9\" (UID: \"d58f73f8-d663-4ec6-b441-1f19834c37a9\") "
May 28 17:54:48.063353 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:48.063326    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d58f73f8-d663-4ec6-b441-1f19834c37a9-kube-api-access-bn9jg" (OuterVolumeSpecName: "kube-api-access-bn9jg") pod "d58f73f8-d663-4ec6-b441-1f19834c37a9" (UID: "d58f73f8-d663-4ec6-b441-1f19834c37a9"). InnerVolumeSpecName "kube-api-access-bn9jg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:54:48.162093 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:48.162054    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn9jg\" (UniqueName: \"kubernetes.io/projected/d58f73f8-d663-4ec6-b441-1f19834c37a9-kube-api-access-bn9jg\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:54:48.675074 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:48.675038    2560 generic.go:358] "Generic (PLEG): container finished" podID="d58f73f8-d663-4ec6-b441-1f19834c37a9" containerID="450228fa17f07bfb52329c070a946a60f6efdab65b22f5f7133521f32ba9e812" exitCode=0
May 28 17:54:48.675245 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:48.675093    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-l57h8"
May 28 17:54:48.675245 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:48.675127    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-l57h8" event={"ID":"d58f73f8-d663-4ec6-b441-1f19834c37a9","Type":"ContainerDied","Data":"450228fa17f07bfb52329c070a946a60f6efdab65b22f5f7133521f32ba9e812"}
May 28 17:54:48.675245 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:48.675162    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-l57h8" event={"ID":"d58f73f8-d663-4ec6-b441-1f19834c37a9","Type":"ContainerDied","Data":"873a6d100f94afcdee1a3151dcb17963c7311490248c9b2e5201b77af4c200b3"}
May 28 17:54:48.675245 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:48.675181    2560 scope.go:117] "RemoveContainer" containerID="450228fa17f07bfb52329c070a946a60f6efdab65b22f5f7133521f32ba9e812"
May 28 17:54:48.683123 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:48.683105    2560 scope.go:117] "RemoveContainer" containerID="450228fa17f07bfb52329c070a946a60f6efdab65b22f5f7133521f32ba9e812"
May 28 17:54:48.683383 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:54:48.683363    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"450228fa17f07bfb52329c070a946a60f6efdab65b22f5f7133521f32ba9e812\": container with ID starting with 450228fa17f07bfb52329c070a946a60f6efdab65b22f5f7133521f32ba9e812 not found: ID does not exist" containerID="450228fa17f07bfb52329c070a946a60f6efdab65b22f5f7133521f32ba9e812"
May 28 17:54:48.683457 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:48.683390    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"450228fa17f07bfb52329c070a946a60f6efdab65b22f5f7133521f32ba9e812"} err="failed to get container status \"450228fa17f07bfb52329c070a946a60f6efdab65b22f5f7133521f32ba9e812\": rpc error: code = NotFound desc = could not find container \"450228fa17f07bfb52329c070a946a60f6efdab65b22f5f7133521f32ba9e812\": container with ID starting with 450228fa17f07bfb52329c070a946a60f6efdab65b22f5f7133521f32ba9e812 not found: ID does not exist"
May 28 17:54:48.694419 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:48.694396    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-l57h8"]
May 28 17:54:48.697944 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:48.697911    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-l57h8"]
May 28 17:54:49.517951 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:49.517893    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8f443a0-3fd4-43bd-ad7c-62170c213f11" path="/var/lib/kubelet/pods/a8f443a0-3fd4-43bd-ad7c-62170c213f11/volumes"
May 28 17:54:49.518398 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:49.518162    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d58f73f8-d663-4ec6-b441-1f19834c37a9" path="/var/lib/kubelet/pods/d58f73f8-d663-4ec6-b441-1f19834c37a9/volumes"
May 28 17:54:49.518443 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:54:49.518429    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779" path="/var/lib/kubelet/pods/fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779/volumes"
May 28 17:55:52.383275 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.383245    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"]
May 28 17:55:52.383746 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.383522    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779" containerName="authorino"
May 28 17:55:52.383746 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.383533    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779" containerName="authorino"
May 28 17:55:52.383746 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.383544    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58f73f8-d663-4ec6-b441-1f19834c37a9" containerName="authorino"
May 28 17:55:52.383746 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.383549    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58f73f8-d663-4ec6-b441-1f19834c37a9" containerName="authorino"
May 28 17:55:52.383746 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.383600    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="d58f73f8-d663-4ec6-b441-1f19834c37a9" containerName="authorino"
May 28 17:55:52.383746 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.383607    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="fa5bb4b7-0b76-4f1c-8f4e-5ac09ac66779" containerName="authorino"
May 28 17:55:52.386649 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.386629    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.389087 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.389056    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
May 28 17:55:52.389223 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.389056    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
May 28 17:55:52.390031 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.390013    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
May 28 17:55:52.390133 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.390061    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7wjbl\""
May 28 17:55:52.396598 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.396574    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"]
May 28 17:55:52.477645 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.477608    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48d40739-c449-4207-864a-44f3ee69b2fc-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.477645 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.477649    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48d40739-c449-4207-864a-44f3ee69b2fc-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.477857 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.477778    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48d40739-c449-4207-864a-44f3ee69b2fc-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.477857 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.477822    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh5gj\" (UniqueName: \"kubernetes.io/projected/48d40739-c449-4207-864a-44f3ee69b2fc-kube-api-access-bh5gj\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.477947 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.477856    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/48d40739-c449-4207-864a-44f3ee69b2fc-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.477947 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.477890    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48d40739-c449-4207-864a-44f3ee69b2fc-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.579172 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.579136    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48d40739-c449-4207-864a-44f3ee69b2fc-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.579350 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.579199    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48d40739-c449-4207-864a-44f3ee69b2fc-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.579350 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.579221    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bh5gj\" (UniqueName: \"kubernetes.io/projected/48d40739-c449-4207-864a-44f3ee69b2fc-kube-api-access-bh5gj\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.579350 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.579260    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/48d40739-c449-4207-864a-44f3ee69b2fc-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.579350 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.579294    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48d40739-c449-4207-864a-44f3ee69b2fc-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.579519 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.579358    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48d40739-c449-4207-864a-44f3ee69b2fc-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.579606 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.579586    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48d40739-c449-4207-864a-44f3ee69b2fc-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.579688 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.579664    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48d40739-c449-4207-864a-44f3ee69b2fc-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.579726 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.579675    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/48d40739-c449-4207-864a-44f3ee69b2fc-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.581598 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.581580    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48d40739-c449-4207-864a-44f3ee69b2fc-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.581909 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.581892    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48d40739-c449-4207-864a-44f3ee69b2fc-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.586442 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.586419    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh5gj\" (UniqueName: \"kubernetes.io/projected/48d40739-c449-4207-864a-44f3ee69b2fc-kube-api-access-bh5gj\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-2kknq\" (UID: \"48d40739-c449-4207-864a-44f3ee69b2fc\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.696760 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.696652    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:55:52.823341 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.823307    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"]
May 28 17:55:52.826508 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:55:52.826481    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48d40739_c449_4207_864a_44f3ee69b2fc.slice/crio-a8a834e2eb3f9df11377181d0a3988eae3c18c5e1ece34b2494d99d293a48044 WatchSource:0}: Error finding container a8a834e2eb3f9df11377181d0a3988eae3c18c5e1ece34b2494d99d293a48044: Status 404 returned error can't find the container with id a8a834e2eb3f9df11377181d0a3988eae3c18c5e1ece34b2494d99d293a48044
May 28 17:55:52.879637 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:52.879602    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq" event={"ID":"48d40739-c449-4207-864a-44f3ee69b2fc","Type":"ContainerStarted","Data":"a8a834e2eb3f9df11377181d0a3988eae3c18c5e1ece34b2494d99d293a48044"}
May 28 17:55:58.684293 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.679360    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"]
May 28 17:55:58.726511 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.726464    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"]
May 28 17:55:58.726691 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.726535    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.729179 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.729155    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
May 28 17:55:58.838188 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.838163    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.838307 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.838199    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw4rf\" (UniqueName: \"kubernetes.io/projected/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-kube-api-access-vw4rf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.838307 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.838250    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.838421 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.838359    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.838421 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.838407    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.838485 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.838437    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.901395 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.901347    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq" event={"ID":"48d40739-c449-4207-864a-44f3ee69b2fc","Type":"ContainerStarted","Data":"c7285b016b994a752ad2861c51f879f69cdd5b9f09eafd9164f186e48b725266"}
May 28 17:55:58.939680 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.939596    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vw4rf\" (UniqueName: \"kubernetes.io/projected/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-kube-api-access-vw4rf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.939680 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.939673    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.939870 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.939721    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.939870 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.939751    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.939870 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.939780    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.939870 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.939821    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.940252 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.940195    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.940385 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.940263    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.940385 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.940270    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.942249 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.942226    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.942710 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.942688    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:58.948229 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:58.948198    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw4rf\" (UniqueName: \"kubernetes.io/projected/04b2bed1-aa33-4bc1-a6c5-091eceb81f1a-kube-api-access-vw4rf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw\" (UID: \"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:59.037567 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:59.037537    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:55:59.173072 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:59.173048    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"]
May 28 17:55:59.175403 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:55:59.175379    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b2bed1_aa33_4bc1_a6c5_091eceb81f1a.slice/crio-a2a6fc2b11f481d30684007b8f852e10eb26d1c6c0dd8383e7d3fcae2cc3ac42 WatchSource:0}: Error finding container a2a6fc2b11f481d30684007b8f852e10eb26d1c6c0dd8383e7d3fcae2cc3ac42: Status 404 returned error can't find the container with id a2a6fc2b11f481d30684007b8f852e10eb26d1c6c0dd8383e7d3fcae2cc3ac42
May 28 17:55:59.178007 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:59.177988    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 17:55:59.906706 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:59.906665    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw" event={"ID":"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a","Type":"ContainerStarted","Data":"7bcc8fa92f683ca28616e4be0fb43c71961033f69ed12c2b493ec79dbfd43e4a"}
May 28 17:55:59.906706 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:55:59.906708    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw" event={"ID":"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a","Type":"ContainerStarted","Data":"a2a6fc2b11f481d30684007b8f852e10eb26d1c6c0dd8383e7d3fcae2cc3ac42"}
May 28 17:56:04.924693 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:04.924656    2560 generic.go:358] "Generic (PLEG): container finished" podID="04b2bed1-aa33-4bc1-a6c5-091eceb81f1a" containerID="7bcc8fa92f683ca28616e4be0fb43c71961033f69ed12c2b493ec79dbfd43e4a" exitCode=0
May 28 17:56:04.925106 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:04.924733    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw" event={"ID":"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a","Type":"ContainerDied","Data":"7bcc8fa92f683ca28616e4be0fb43c71961033f69ed12c2b493ec79dbfd43e4a"}
May 28 17:56:06.934149 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:06.934116    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw" event={"ID":"04b2bed1-aa33-4bc1-a6c5-091eceb81f1a","Type":"ContainerStarted","Data":"23f652825d85b94c00dbb5e117af0a992785d7079f903b46a356395f7737c67b"}
May 28 17:56:06.934577 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:06.934336    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:56:06.952585 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:06.952532    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw" podStartSLOduration=7.643562176 podStartE2EDuration="8.952519909s" podCreationTimestamp="2026-05-28 17:55:58 +0000 UTC" firstStartedPulling="2026-05-28 17:56:04.925406993 +0000 UTC m=+766.029102690" lastFinishedPulling="2026-05-28 17:56:06.234364713 +0000 UTC m=+767.338060423" observedRunningTime="2026-05-28 17:56:06.95182206 +0000 UTC m=+768.055517779" watchObservedRunningTime="2026-05-28 17:56:06.952519909 +0000 UTC m=+768.056215626"
May 28 17:56:07.939514 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:07.939425    2560 generic.go:358] "Generic (PLEG): container finished" podID="48d40739-c449-4207-864a-44f3ee69b2fc" containerID="c7285b016b994a752ad2861c51f879f69cdd5b9f09eafd9164f186e48b725266" exitCode=0
May 28 17:56:07.939514 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:07.939500    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq" event={"ID":"48d40739-c449-4207-864a-44f3ee69b2fc","Type":"ContainerDied","Data":"c7285b016b994a752ad2861c51f879f69cdd5b9f09eafd9164f186e48b725266"}
May 28 17:56:08.944803 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:08.944757    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq" event={"ID":"48d40739-c449-4207-864a-44f3ee69b2fc","Type":"ContainerStarted","Data":"0f3fd20aac86cf162e1ef8a711b9388f28a8374eb4f4b6a9adc806e706103a85"}
May 28 17:56:08.945251 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:08.944984    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:56:08.964373 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:08.964319    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq" podStartSLOduration=1.586645841 podStartE2EDuration="16.964301827s" podCreationTimestamp="2026-05-28 17:55:52 +0000 UTC" firstStartedPulling="2026-05-28 17:55:52.828654389 +0000 UTC m=+753.932350098" lastFinishedPulling="2026-05-28 17:56:08.20631038 +0000 UTC m=+769.310006084" observedRunningTime="2026-05-28 17:56:08.961855171 +0000 UTC m=+770.065550888" watchObservedRunningTime="2026-05-28 17:56:08.964301827 +0000 UTC m=+770.067997544"
May 28 17:56:17.952469 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:17.952434    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw"
May 28 17:56:19.962299 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:19.962270    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-2kknq"
May 28 17:56:36.681367 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:36.681194    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d8b7d4fd5-226bn"]
May 28 17:56:36.681896 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:36.681675    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-d8b7d4fd5-226bn" podUID="ea459310-6743-487c-987e-efc530bd283f" containerName="authorino" containerID="cri-o://17edc499e5bec8682b9f561ca20fc6e397037910763bc1c857b7de045bc445d9" gracePeriod=30
May 28 17:56:36.929067 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:36.929046    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8b7d4fd5-226bn"
May 28 17:56:36.970661 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:36.970571    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxtmc\" (UniqueName: \"kubernetes.io/projected/ea459310-6743-487c-987e-efc530bd283f-kube-api-access-fxtmc\") pod \"ea459310-6743-487c-987e-efc530bd283f\" (UID: \"ea459310-6743-487c-987e-efc530bd283f\") "
May 28 17:56:36.970661 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:36.970637    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ea459310-6743-487c-987e-efc530bd283f-tls-cert\") pod \"ea459310-6743-487c-987e-efc530bd283f\" (UID: \"ea459310-6743-487c-987e-efc530bd283f\") "
May 28 17:56:36.972864 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:36.972831    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea459310-6743-487c-987e-efc530bd283f-kube-api-access-fxtmc" (OuterVolumeSpecName: "kube-api-access-fxtmc") pod "ea459310-6743-487c-987e-efc530bd283f" (UID: "ea459310-6743-487c-987e-efc530bd283f"). InnerVolumeSpecName "kube-api-access-fxtmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:56:36.981230 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:36.981193    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea459310-6743-487c-987e-efc530bd283f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ea459310-6743-487c-987e-efc530bd283f" (UID: "ea459310-6743-487c-987e-efc530bd283f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 17:56:37.034880 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.034838    2560 generic.go:358] "Generic (PLEG): container finished" podID="ea459310-6743-487c-987e-efc530bd283f" containerID="17edc499e5bec8682b9f561ca20fc6e397037910763bc1c857b7de045bc445d9" exitCode=0
May 28 17:56:37.035106 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.034890    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8b7d4fd5-226bn" event={"ID":"ea459310-6743-487c-987e-efc530bd283f","Type":"ContainerDied","Data":"17edc499e5bec8682b9f561ca20fc6e397037910763bc1c857b7de045bc445d9"}
May 28 17:56:37.035106 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.034903    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8b7d4fd5-226bn"
May 28 17:56:37.035106 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.034914    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8b7d4fd5-226bn" event={"ID":"ea459310-6743-487c-987e-efc530bd283f","Type":"ContainerDied","Data":"347abc0f9f50eb610b34316971d7a143f68b68409abbba7abf9f73bee335df43"}
May 28 17:56:37.035106 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.034953    2560 scope.go:117] "RemoveContainer" containerID="17edc499e5bec8682b9f561ca20fc6e397037910763bc1c857b7de045bc445d9"
May 28 17:56:37.043379 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.043354    2560 scope.go:117] "RemoveContainer" containerID="17edc499e5bec8682b9f561ca20fc6e397037910763bc1c857b7de045bc445d9"
May 28 17:56:37.043659 ip-10-0-135-41 kubenswrapper[2560]: E0528 17:56:37.043641    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17edc499e5bec8682b9f561ca20fc6e397037910763bc1c857b7de045bc445d9\": container with ID starting with 17edc499e5bec8682b9f561ca20fc6e397037910763bc1c857b7de045bc445d9 not found: ID does not exist" containerID="17edc499e5bec8682b9f561ca20fc6e397037910763bc1c857b7de045bc445d9"
May 28 17:56:37.043724 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.043672    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17edc499e5bec8682b9f561ca20fc6e397037910763bc1c857b7de045bc445d9"} err="failed to get container status \"17edc499e5bec8682b9f561ca20fc6e397037910763bc1c857b7de045bc445d9\": rpc error: code = NotFound desc = could not find container \"17edc499e5bec8682b9f561ca20fc6e397037910763bc1c857b7de045bc445d9\": container with ID starting with 17edc499e5bec8682b9f561ca20fc6e397037910763bc1c857b7de045bc445d9 not found: ID does not exist"
May 28 17:56:37.060718 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.060688    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d8b7d4fd5-226bn"]
May 28 17:56:37.067121 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.067094    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-d8b7d4fd5-226bn"]
May 28 17:56:37.071722 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.071692    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxtmc\" (UniqueName: \"kubernetes.io/projected/ea459310-6743-487c-987e-efc530bd283f-kube-api-access-fxtmc\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:56:37.071722 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.071715    2560 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ea459310-6743-487c-987e-efc530bd283f-tls-cert\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 17:56:37.517689 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.517656    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea459310-6743-487c-987e-efc530bd283f" path="/var/lib/kubelet/pods/ea459310-6743-487c-987e-efc530bd283f/volumes"
May 28 17:56:37.691655 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.691619    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"]
May 28 17:56:37.692102 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.691916    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea459310-6743-487c-987e-efc530bd283f" containerName="authorino"
May 28 17:56:37.692102 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.691945    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea459310-6743-487c-987e-efc530bd283f" containerName="authorino"
May 28 17:56:37.692102 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.692006    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="ea459310-6743-487c-987e-efc530bd283f" containerName="authorino"
May 28 17:56:37.695333 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.695307    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.697857 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.697835    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
May 28 17:56:37.710918 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.710889    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"]
May 28 17:56:37.777987 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.777964    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.778094 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.778004    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.778094 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.778044    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.778094 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.778067    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njm5g\" (UniqueName: \"kubernetes.io/projected/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-kube-api-access-njm5g\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.778094 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.778089    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.778242 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.778109    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.879217 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.879180    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.879375 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.879234    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.879375 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.879259    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-njm5g\" (UniqueName: \"kubernetes.io/projected/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-kube-api-access-njm5g\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.879375 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.879290    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.879375 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.879311    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.879375 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.879353    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.879724 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.879703    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.879786 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.879729    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.879786 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.879774    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.881645 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.881615    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.881701 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.881634    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:37.887250 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:37.887227    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njm5g\" (UniqueName: \"kubernetes.io/projected/fe27ccc1-e1e8-4d29-b8db-12caa0684c43-kube-api-access-njm5g\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct\" (UID: \"fe27ccc1-e1e8-4d29-b8db-12caa0684c43\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:38.006339 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:38.006306    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:38.140268 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:38.140120    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"]
May 28 17:56:38.142978 ip-10-0-135-41 kubenswrapper[2560]: W0528 17:56:38.142948    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe27ccc1_e1e8_4d29_b8db_12caa0684c43.slice/crio-5c54fdccc45054b24d7a8fc0a0751931663a55971e48a20275b36b042eb5d68d WatchSource:0}: Error finding container 5c54fdccc45054b24d7a8fc0a0751931663a55971e48a20275b36b042eb5d68d: Status 404 returned error can't find the container with id 5c54fdccc45054b24d7a8fc0a0751931663a55971e48a20275b36b042eb5d68d
May 28 17:56:39.046981 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:39.046920    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct" event={"ID":"fe27ccc1-e1e8-4d29-b8db-12caa0684c43","Type":"ContainerStarted","Data":"10c4774e8d458fd867693d454e6615fc4035e4cdd11c152a047d0cbe2cbfdbc8"}
May 28 17:56:39.046981 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:39.046985    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct" event={"ID":"fe27ccc1-e1e8-4d29-b8db-12caa0684c43","Type":"ContainerStarted","Data":"5c54fdccc45054b24d7a8fc0a0751931663a55971e48a20275b36b042eb5d68d"}
May 28 17:56:44.064519 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:44.064484    2560 generic.go:358] "Generic (PLEG): container finished" podID="fe27ccc1-e1e8-4d29-b8db-12caa0684c43" containerID="10c4774e8d458fd867693d454e6615fc4035e4cdd11c152a047d0cbe2cbfdbc8" exitCode=0
May 28 17:56:44.064914 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:44.064543    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct" event={"ID":"fe27ccc1-e1e8-4d29-b8db-12caa0684c43","Type":"ContainerDied","Data":"10c4774e8d458fd867693d454e6615fc4035e4cdd11c152a047d0cbe2cbfdbc8"}
May 28 17:56:45.069830 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:45.069793    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct" event={"ID":"fe27ccc1-e1e8-4d29-b8db-12caa0684c43","Type":"ContainerStarted","Data":"5457efd56faf1ccbcf127c9adf5253395e9b7d14be5b64fc8d4d31fa6ee86567"}
May 28 17:56:45.070328 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:45.070050    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 17:56:45.088151 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:45.088084    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct" podStartSLOduration=7.826327908 podStartE2EDuration="8.08806242s" podCreationTimestamp="2026-05-28 17:56:37 +0000 UTC" firstStartedPulling="2026-05-28 17:56:44.06517691 +0000 UTC m=+805.168872606" lastFinishedPulling="2026-05-28 17:56:44.32691142 +0000 UTC m=+805.430607118" observedRunningTime="2026-05-28 17:56:45.086571024 +0000 UTC m=+806.190266742" watchObservedRunningTime="2026-05-28 17:56:45.08806242 +0000 UTC m=+806.191758139"
May 28 17:56:56.086962 ip-10-0-135-41 kubenswrapper[2560]: I0528 17:56:56.086903    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct"
May 28 18:00:00.144077 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:00.143993    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29666520-5c5vb"]
May 28 18:00:00.147332 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:00.147315    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666520-5c5vb"
May 28 18:00:00.149874 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:00.149857    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-k2j7x\""
May 28 18:00:00.154905 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:00.154884    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29666520-5c5vb"]
May 28 18:00:00.300751 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:00.300711    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stfjj\" (UniqueName: \"kubernetes.io/projected/f04bd314-2659-4689-b765-9b997e63a2bf-kube-api-access-stfjj\") pod \"maas-api-key-cleanup-29666520-5c5vb\" (UID: \"f04bd314-2659-4689-b765-9b997e63a2bf\") " pod="opendatahub/maas-api-key-cleanup-29666520-5c5vb"
May 28 18:00:00.401570 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:00.401487    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-stfjj\" (UniqueName: \"kubernetes.io/projected/f04bd314-2659-4689-b765-9b997e63a2bf-kube-api-access-stfjj\") pod \"maas-api-key-cleanup-29666520-5c5vb\" (UID: \"f04bd314-2659-4689-b765-9b997e63a2bf\") " pod="opendatahub/maas-api-key-cleanup-29666520-5c5vb"
May 28 18:00:00.410650 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:00.410618    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stfjj\" (UniqueName: \"kubernetes.io/projected/f04bd314-2659-4689-b765-9b997e63a2bf-kube-api-access-stfjj\") pod \"maas-api-key-cleanup-29666520-5c5vb\" (UID: \"f04bd314-2659-4689-b765-9b997e63a2bf\") " pod="opendatahub/maas-api-key-cleanup-29666520-5c5vb"
May 28 18:00:00.459044 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:00.459002    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666520-5c5vb"
May 28 18:00:00.590469 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:00.590387    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29666520-5c5vb"]
May 28 18:00:00.593157 ip-10-0-135-41 kubenswrapper[2560]: W0528 18:00:00.593130    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf04bd314_2659_4689_b765_9b997e63a2bf.slice/crio-87941eae0e3b19f31da0dabb0f7ce5efcaeed5474a6fdea9a3edc901f17bd9a0 WatchSource:0}: Error finding container 87941eae0e3b19f31da0dabb0f7ce5efcaeed5474a6fdea9a3edc901f17bd9a0: Status 404 returned error can't find the container with id 87941eae0e3b19f31da0dabb0f7ce5efcaeed5474a6fdea9a3edc901f17bd9a0
May 28 18:00:00.692052 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:00.691970    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666520-5c5vb" event={"ID":"f04bd314-2659-4689-b765-9b997e63a2bf","Type":"ContainerStarted","Data":"87941eae0e3b19f31da0dabb0f7ce5efcaeed5474a6fdea9a3edc901f17bd9a0"}
May 28 18:00:05.711028 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:05.710948    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666520-5c5vb" event={"ID":"f04bd314-2659-4689-b765-9b997e63a2bf","Type":"ContainerStarted","Data":"99b73d5cb9f17e5d0ada992c73f6ccdb41fdfe1b254c2d11b84f9d45678dbcd8"}
May 28 18:00:05.728903 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:05.728847    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29666520-5c5vb" podStartSLOduration=1.188168807 podStartE2EDuration="5.728832352s" podCreationTimestamp="2026-05-28 18:00:00 +0000 UTC" firstStartedPulling="2026-05-28 18:00:00.595352485 +0000 UTC m=+1001.699048181" lastFinishedPulling="2026-05-28 18:00:05.136016027 +0000 UTC m=+1006.239711726" observedRunningTime="2026-05-28 18:00:05.727685851 +0000 UTC m=+1006.831381570" watchObservedRunningTime="2026-05-28 18:00:05.728832352 +0000 UTC m=+1006.832528070"
May 28 18:00:06.716289 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:06.716254    2560 generic.go:358] "Generic (PLEG): container finished" podID="f04bd314-2659-4689-b765-9b997e63a2bf" containerID="99b73d5cb9f17e5d0ada992c73f6ccdb41fdfe1b254c2d11b84f9d45678dbcd8" exitCode=0
May 28 18:00:06.716679 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:06.716319    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666520-5c5vb" event={"ID":"f04bd314-2659-4689-b765-9b997e63a2bf","Type":"ContainerDied","Data":"99b73d5cb9f17e5d0ada992c73f6ccdb41fdfe1b254c2d11b84f9d45678dbcd8"}
May 28 18:00:07.852308 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:07.852282    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666520-5c5vb"
May 28 18:00:07.866764 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:07.866729    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stfjj\" (UniqueName: \"kubernetes.io/projected/f04bd314-2659-4689-b765-9b997e63a2bf-kube-api-access-stfjj\") pod \"f04bd314-2659-4689-b765-9b997e63a2bf\" (UID: \"f04bd314-2659-4689-b765-9b997e63a2bf\") "
May 28 18:00:07.869511 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:07.869472    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f04bd314-2659-4689-b765-9b997e63a2bf-kube-api-access-stfjj" (OuterVolumeSpecName: "kube-api-access-stfjj") pod "f04bd314-2659-4689-b765-9b997e63a2bf" (UID: "f04bd314-2659-4689-b765-9b997e63a2bf"). InnerVolumeSpecName "kube-api-access-stfjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 18:00:07.968195 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:07.968158    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stfjj\" (UniqueName: \"kubernetes.io/projected/f04bd314-2659-4689-b765-9b997e63a2bf-kube-api-access-stfjj\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 18:00:08.724018 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:08.723979    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666520-5c5vb" event={"ID":"f04bd314-2659-4689-b765-9b997e63a2bf","Type":"ContainerDied","Data":"87941eae0e3b19f31da0dabb0f7ce5efcaeed5474a6fdea9a3edc901f17bd9a0"}
May 28 18:00:08.724018 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:08.724015    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87941eae0e3b19f31da0dabb0f7ce5efcaeed5474a6fdea9a3edc901f17bd9a0"
May 28 18:00:08.724221 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:00:08.724032    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666520-5c5vb"
May 28 18:15:00.130617 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:00.130541    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29666535-z5xg2"]
May 28 18:15:00.131137 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:00.130842    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f04bd314-2659-4689-b765-9b997e63a2bf" containerName="cleanup"
May 28 18:15:00.131137 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:00.130853    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f04bd314-2659-4689-b765-9b997e63a2bf" containerName="cleanup"
May 28 18:15:00.131137 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:00.130909    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="f04bd314-2659-4689-b765-9b997e63a2bf" containerName="cleanup"
May 28 18:15:00.133691 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:00.133676    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666535-z5xg2"
May 28 18:15:00.135888 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:00.135866    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-k2j7x\""
May 28 18:15:00.148113 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:00.148089    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29666535-z5xg2"]
May 28 18:15:00.290060 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:00.290028    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssbcj\" (UniqueName: \"kubernetes.io/projected/7e795b25-3857-4af2-b00e-2d23e1e4e617-kube-api-access-ssbcj\") pod \"maas-api-key-cleanup-29666535-z5xg2\" (UID: \"7e795b25-3857-4af2-b00e-2d23e1e4e617\") " pod="opendatahub/maas-api-key-cleanup-29666535-z5xg2"
May 28 18:15:00.390763 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:00.390676    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ssbcj\" (UniqueName: \"kubernetes.io/projected/7e795b25-3857-4af2-b00e-2d23e1e4e617-kube-api-access-ssbcj\") pod \"maas-api-key-cleanup-29666535-z5xg2\" (UID: \"7e795b25-3857-4af2-b00e-2d23e1e4e617\") " pod="opendatahub/maas-api-key-cleanup-29666535-z5xg2"
May 28 18:15:00.398967 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:00.398922    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssbcj\" (UniqueName: \"kubernetes.io/projected/7e795b25-3857-4af2-b00e-2d23e1e4e617-kube-api-access-ssbcj\") pod \"maas-api-key-cleanup-29666535-z5xg2\" (UID: \"7e795b25-3857-4af2-b00e-2d23e1e4e617\") " pod="opendatahub/maas-api-key-cleanup-29666535-z5xg2"
May 28 18:15:00.443895 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:00.443854    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666535-z5xg2"
May 28 18:15:00.577184 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:00.577157    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29666535-z5xg2"]
May 28 18:15:00.579898 ip-10-0-135-41 kubenswrapper[2560]: W0528 18:15:00.579869    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e795b25_3857_4af2_b00e_2d23e1e4e617.slice/crio-44428e06a459e392bf29d90f932e3bbb0c5d0541a7d693868093c75019870f2e WatchSource:0}: Error finding container 44428e06a459e392bf29d90f932e3bbb0c5d0541a7d693868093c75019870f2e: Status 404 returned error can't find the container with id 44428e06a459e392bf29d90f932e3bbb0c5d0541a7d693868093c75019870f2e
May 28 18:15:00.581678 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:00.581661    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 18:15:00.642421 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:00.642329    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666535-z5xg2" event={"ID":"7e795b25-3857-4af2-b00e-2d23e1e4e617","Type":"ContainerStarted","Data":"44428e06a459e392bf29d90f932e3bbb0c5d0541a7d693868093c75019870f2e"}
May 28 18:15:01.646544 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:01.646510    2560 generic.go:358] "Generic (PLEG): container finished" podID="7e795b25-3857-4af2-b00e-2d23e1e4e617" containerID="3cb9515a06f6fdc0b48354b6f327721f66eb5b4b95331e96f5fa4f6045be766f" exitCode=0
May 28 18:15:01.646996 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:01.646551    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666535-z5xg2" event={"ID":"7e795b25-3857-4af2-b00e-2d23e1e4e617","Type":"ContainerDied","Data":"3cb9515a06f6fdc0b48354b6f327721f66eb5b4b95331e96f5fa4f6045be766f"}
May 28 18:15:02.775412 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:02.775387    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666535-z5xg2"
May 28 18:15:02.914593 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:02.914502    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssbcj\" (UniqueName: \"kubernetes.io/projected/7e795b25-3857-4af2-b00e-2d23e1e4e617-kube-api-access-ssbcj\") pod \"7e795b25-3857-4af2-b00e-2d23e1e4e617\" (UID: \"7e795b25-3857-4af2-b00e-2d23e1e4e617\") "
May 28 18:15:02.916774 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:02.916751    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e795b25-3857-4af2-b00e-2d23e1e4e617-kube-api-access-ssbcj" (OuterVolumeSpecName: "kube-api-access-ssbcj") pod "7e795b25-3857-4af2-b00e-2d23e1e4e617" (UID: "7e795b25-3857-4af2-b00e-2d23e1e4e617"). InnerVolumeSpecName "kube-api-access-ssbcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 18:15:03.015216 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:03.015179    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssbcj\" (UniqueName: \"kubernetes.io/projected/7e795b25-3857-4af2-b00e-2d23e1e4e617-kube-api-access-ssbcj\") on node \"ip-10-0-135-41.ec2.internal\" DevicePath \"\""
May 28 18:15:03.654697 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:03.654653    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666535-z5xg2" event={"ID":"7e795b25-3857-4af2-b00e-2d23e1e4e617","Type":"ContainerDied","Data":"44428e06a459e392bf29d90f932e3bbb0c5d0541a7d693868093c75019870f2e"}
May 28 18:15:03.654697 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:03.654693    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44428e06a459e392bf29d90f932e3bbb0c5d0541a7d693868093c75019870f2e"
May 28 18:15:03.654697 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:15:03.654668    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666535-z5xg2"
May 28 18:19:09.161869 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:09.161797    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29666520-5c5vb_f04bd314-2659-4689-b765-9b997e63a2bf/cleanup/0.log"
May 28 18:19:09.270147 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:09.270119    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29666535-z5xg2_7e795b25-3857-4af2-b00e-2d23e1e4e617/cleanup/0.log"
May 28 18:19:09.629909 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:09.629877    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65fbbfd459-2fsr8_0970116a-e87a-4db0-8dd2-d2eb5f64f236/manager/0.log"
May 28 18:19:09.964032 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:09.963961    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-k2299_e44f759f-b13e-4295-b282-658fff395205/postgres/0.log"
May 28 18:19:11.333663 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:11.333628    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-w5w9c_d381a90c-4d7c-4823-9e12-28fbc9a90ab4/manager/0.log"
May 28 18:19:11.662134 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:11.662056    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-97fp7_039c2a50-474c-425d-9794-70ac390c81da/registry-server/0.log"
May 28 18:19:12.352614 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:12.352585    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4b424c8_33a6915a-2203-4ece-92ff-235f6e7f826b/istio-proxy/0.log"
May 28 18:19:12.818817 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:12.818774    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-7tpzj_214a7f13-c84e-42d1-b854-a8cc63b9de71/istio-proxy/0.log"
May 28 18:19:13.515115 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:13.515081    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct_fe27ccc1-e1e8-4d29-b8db-12caa0684c43/storage-initializer/0.log"
May 28 18:19:13.521273 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:13.521252    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-zpfct_fe27ccc1-e1e8-4d29-b8db-12caa0684c43/main/0.log"
May 28 18:19:13.759502 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:13.759475    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw_04b2bed1-aa33-4bc1-a6c5-091eceb81f1a/storage-initializer/0.log"
May 28 18:19:13.765713 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:13.765654    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdcchgzbw_04b2bed1-aa33-4bc1-a6c5-091eceb81f1a/main/0.log"
May 28 18:19:13.996426 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:13.996395    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-2kknq_48d40739-c449-4207-864a-44f3ee69b2fc/storage-initializer/0.log"
May 28 18:19:14.003249 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:14.003229    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-2kknq_48d40739-c449-4207-864a-44f3ee69b2fc/main/0.log"
May 28 18:19:17.696070 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:17.696033    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v62rz/must-gather-bd2vr"]
May 28 18:19:17.696460 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:17.696327    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e795b25-3857-4af2-b00e-2d23e1e4e617" containerName="cleanup"
May 28 18:19:17.696460 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:17.696337    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e795b25-3857-4af2-b00e-2d23e1e4e617" containerName="cleanup"
May 28 18:19:17.696460 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:17.696394    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="7e795b25-3857-4af2-b00e-2d23e1e4e617" containerName="cleanup"
May 28 18:19:17.699331 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:17.699310    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v62rz/must-gather-bd2vr"
May 28 18:19:17.703575 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:17.703556    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-v62rz\"/\"default-dockercfg-kgtl6\""
May 28 18:19:17.703891 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:17.703875    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v62rz\"/\"kube-root-ca.crt\""
May 28 18:19:17.704594 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:17.704579    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v62rz\"/\"openshift-service-ca.crt\""
May 28 18:19:17.726388 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:17.726351    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v62rz/must-gather-bd2vr"]
May 28 18:19:17.778824 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:17.778779    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6w5c\" (UniqueName: \"kubernetes.io/projected/bd1109d2-bd61-4d3f-916c-b9a8fdd2494f-kube-api-access-f6w5c\") pod \"must-gather-bd2vr\" (UID: \"bd1109d2-bd61-4d3f-916c-b9a8fdd2494f\") " pod="openshift-must-gather-v62rz/must-gather-bd2vr"
May 28 18:19:17.779019 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:17.778831    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/bd1109d2-bd61-4d3f-916c-b9a8fdd2494f-must-gather-output\") pod \"must-gather-bd2vr\" (UID: \"bd1109d2-bd61-4d3f-916c-b9a8fdd2494f\") " pod="openshift-must-gather-v62rz/must-gather-bd2vr"
May 28 18:19:17.879800 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:17.879759    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f6w5c\" (UniqueName: \"kubernetes.io/projected/bd1109d2-bd61-4d3f-916c-b9a8fdd2494f-kube-api-access-f6w5c\") pod \"must-gather-bd2vr\" (UID: \"bd1109d2-bd61-4d3f-916c-b9a8fdd2494f\") " pod="openshift-must-gather-v62rz/must-gather-bd2vr"
May 28 18:19:17.880015 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:17.879820    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/bd1109d2-bd61-4d3f-916c-b9a8fdd2494f-must-gather-output\") pod \"must-gather-bd2vr\" (UID: \"bd1109d2-bd61-4d3f-916c-b9a8fdd2494f\") " pod="openshift-must-gather-v62rz/must-gather-bd2vr"
May 28 18:19:17.880164 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:17.880146    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/bd1109d2-bd61-4d3f-916c-b9a8fdd2494f-must-gather-output\") pod \"must-gather-bd2vr\" (UID: \"bd1109d2-bd61-4d3f-916c-b9a8fdd2494f\") " pod="openshift-must-gather-v62rz/must-gather-bd2vr"
May 28 18:19:17.888585 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:17.888560    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6w5c\" (UniqueName: \"kubernetes.io/projected/bd1109d2-bd61-4d3f-916c-b9a8fdd2494f-kube-api-access-f6w5c\") pod \"must-gather-bd2vr\" (UID: \"bd1109d2-bd61-4d3f-916c-b9a8fdd2494f\") " pod="openshift-must-gather-v62rz/must-gather-bd2vr"
May 28 18:19:18.008329 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:18.008238    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v62rz/must-gather-bd2vr"
May 28 18:19:18.139044 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:18.138886    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v62rz/must-gather-bd2vr"]
May 28 18:19:18.141825 ip-10-0-135-41 kubenswrapper[2560]: W0528 18:19:18.141794    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd1109d2_bd61_4d3f_916c_b9a8fdd2494f.slice/crio-b15f922f83a243584bd33f14bc19ee2cf837b4f2dce480986100c7e8e36520e4 WatchSource:0}: Error finding container b15f922f83a243584bd33f14bc19ee2cf837b4f2dce480986100c7e8e36520e4: Status 404 returned error can't find the container with id b15f922f83a243584bd33f14bc19ee2cf837b4f2dce480986100c7e8e36520e4
May 28 18:19:18.489484 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:18.489450    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v62rz/must-gather-bd2vr" event={"ID":"bd1109d2-bd61-4d3f-916c-b9a8fdd2494f","Type":"ContainerStarted","Data":"b15f922f83a243584bd33f14bc19ee2cf837b4f2dce480986100c7e8e36520e4"}
May 28 18:19:19.495368 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:19.495328    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v62rz/must-gather-bd2vr" event={"ID":"bd1109d2-bd61-4d3f-916c-b9a8fdd2494f","Type":"ContainerStarted","Data":"e478cba734c92d09a996fbdd553bc10324d5ef9f8a1238976f17ef455226a234"}
May 28 18:19:19.495845 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:19.495385    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v62rz/must-gather-bd2vr" event={"ID":"bd1109d2-bd61-4d3f-916c-b9a8fdd2494f","Type":"ContainerStarted","Data":"47f77cb8a368dfd1f5a83ba9f117c5f1669100fdc9caeaf1cac9487866a3e0dd"}
May 28 18:19:19.513401 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:19.513349    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-v62rz/must-gather-bd2vr" podStartSLOduration=1.6439157739999999 podStartE2EDuration="2.513336133s" podCreationTimestamp="2026-05-28 18:19:17 +0000 UTC" firstStartedPulling="2026-05-28 18:19:18.143532371 +0000 UTC m=+2159.247228069" lastFinishedPulling="2026-05-28 18:19:19.012952732 +0000 UTC m=+2160.116648428" observedRunningTime="2026-05-28 18:19:19.511768763 +0000 UTC m=+2160.615464490" watchObservedRunningTime="2026-05-28 18:19:19.513336133 +0000 UTC m=+2160.617031851"
May 28 18:19:20.647654 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:20.647593    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hgbhs_a8061064-d0c1-4eea-bb4c-2c237c83b89c/global-pull-secret-syncer/0.log"
May 28 18:19:20.766396 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:20.766355    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-96mcv_f94fa363-005b-460a-adea-49116e3b4776/konnectivity-agent/0.log"
May 28 18:19:20.843815 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:20.843788    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-41.ec2.internal_a7836682e045d93471bf2a629158ba14/haproxy/0.log"
May 28 18:19:25.285601 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:25.285561    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-w5w9c_d381a90c-4d7c-4823-9e12-28fbc9a90ab4/manager/0.log"
May 28 18:19:25.370345 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:25.370309    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-97fp7_039c2a50-474c-425d-9794-70ac390c81da/registry-server/0.log"
May 28 18:19:27.459661 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:27.459618    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9b8kj_3a8f9dc1-b903-4ee5-a69d-27f2c1d29320/node-exporter/0.log"
May 28 18:19:27.483714 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:27.483687    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9b8kj_3a8f9dc1-b903-4ee5-a69d-27f2c1d29320/kube-rbac-proxy/0.log"
May 28 18:19:27.503745 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:27.503719    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9b8kj_3a8f9dc1-b903-4ee5-a69d-27f2c1d29320/init-textfile/0.log"
May 28 18:19:29.108922 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.108894    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-27wps_655375f3-42df-4046-b86a-386dfee4ce6d/networking-console-plugin/0.log"
May 28 18:19:29.212188 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.212141    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"]
May 28 18:19:29.217202 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.217162    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.226442 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.226412    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"]
May 28 18:19:29.288808 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.288775    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b4f3204-d53e-4852-833a-10c2afa12d64-lib-modules\") pod \"perf-node-gather-daemonset-xslrl\" (UID: \"5b4f3204-d53e-4852-833a-10c2afa12d64\") " pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.289017 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.288822    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79n7f\" (UniqueName: \"kubernetes.io/projected/5b4f3204-d53e-4852-833a-10c2afa12d64-kube-api-access-79n7f\") pod \"perf-node-gather-daemonset-xslrl\" (UID: \"5b4f3204-d53e-4852-833a-10c2afa12d64\") " pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.289017 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.288861    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5b4f3204-d53e-4852-833a-10c2afa12d64-proc\") pod \"perf-node-gather-daemonset-xslrl\" (UID: \"5b4f3204-d53e-4852-833a-10c2afa12d64\") " pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.289017 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.288898    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b4f3204-d53e-4852-833a-10c2afa12d64-sys\") pod \"perf-node-gather-daemonset-xslrl\" (UID: \"5b4f3204-d53e-4852-833a-10c2afa12d64\") " pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.289017 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.288922    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5b4f3204-d53e-4852-833a-10c2afa12d64-podres\") pod \"perf-node-gather-daemonset-xslrl\" (UID: \"5b4f3204-d53e-4852-833a-10c2afa12d64\") " pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.390446 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.390347    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b4f3204-d53e-4852-833a-10c2afa12d64-sys\") pod \"perf-node-gather-daemonset-xslrl\" (UID: \"5b4f3204-d53e-4852-833a-10c2afa12d64\") " pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.390446 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.390429    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5b4f3204-d53e-4852-833a-10c2afa12d64-podres\") pod \"perf-node-gather-daemonset-xslrl\" (UID: \"5b4f3204-d53e-4852-833a-10c2afa12d64\") " pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.390686 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.390489    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b4f3204-d53e-4852-833a-10c2afa12d64-lib-modules\") pod \"perf-node-gather-daemonset-xslrl\" (UID: \"5b4f3204-d53e-4852-833a-10c2afa12d64\") " pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.390686 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.390550    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-79n7f\" (UniqueName: \"kubernetes.io/projected/5b4f3204-d53e-4852-833a-10c2afa12d64-kube-api-access-79n7f\") pod \"perf-node-gather-daemonset-xslrl\" (UID: \"5b4f3204-d53e-4852-833a-10c2afa12d64\") " pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.390686 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.390597    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5b4f3204-d53e-4852-833a-10c2afa12d64-proc\") pod \"perf-node-gather-daemonset-xslrl\" (UID: \"5b4f3204-d53e-4852-833a-10c2afa12d64\") " pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.390860 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.390740    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5b4f3204-d53e-4852-833a-10c2afa12d64-proc\") pod \"perf-node-gather-daemonset-xslrl\" (UID: \"5b4f3204-d53e-4852-833a-10c2afa12d64\") " pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.390860 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.390807    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b4f3204-d53e-4852-833a-10c2afa12d64-sys\") pod \"perf-node-gather-daemonset-xslrl\" (UID: \"5b4f3204-d53e-4852-833a-10c2afa12d64\") " pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.390994 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.390952    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5b4f3204-d53e-4852-833a-10c2afa12d64-podres\") pod \"perf-node-gather-daemonset-xslrl\" (UID: \"5b4f3204-d53e-4852-833a-10c2afa12d64\") " pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.391073 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.391053    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b4f3204-d53e-4852-833a-10c2afa12d64-lib-modules\") pod \"perf-node-gather-daemonset-xslrl\" (UID: \"5b4f3204-d53e-4852-833a-10c2afa12d64\") " pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.399377 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.399343    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79n7f\" (UniqueName: \"kubernetes.io/projected/5b4f3204-d53e-4852-833a-10c2afa12d64-kube-api-access-79n7f\") pod \"perf-node-gather-daemonset-xslrl\" (UID: \"5b4f3204-d53e-4852-833a-10c2afa12d64\") " pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.533875 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.533838    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:29.705638 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:29.705535    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"]
May 28 18:19:30.546227 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:30.546176    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl" event={"ID":"5b4f3204-d53e-4852-833a-10c2afa12d64","Type":"ContainerStarted","Data":"166b382ad13377ec172413eef57b94df58c11fa63a336e2928320f0e43efdc80"}
May 28 18:19:30.546227 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:30.546233    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl" event={"ID":"5b4f3204-d53e-4852-833a-10c2afa12d64","Type":"ContainerStarted","Data":"fb160d8850c5f82cea4ee183beb58bba729afd800a0b60b7f3386d4a8647bb56"}
May 28 18:19:30.547120 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:30.547089    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:30.563059 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:30.563012    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl" podStartSLOduration=1.562994825 podStartE2EDuration="1.562994825s" podCreationTimestamp="2026-05-28 18:19:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 18:19:30.561543952 +0000 UTC m=+2171.665239670" watchObservedRunningTime="2026-05-28 18:19:30.562994825 +0000 UTC m=+2171.666690585"
May 28 18:19:31.453149 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:31.453116    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cf92m_b8a6a83f-2e25-446a-8bac-f1c1e9f4defc/dns/0.log"
May 28 18:19:31.473768 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:31.473739    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cf92m_b8a6a83f-2e25-446a-8bac-f1c1e9f4defc/kube-rbac-proxy/0.log"
May 28 18:19:31.606068 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:31.606034    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5mv2t_70ec7ff0-1089-4ca1-8238-da94d4b1c031/dns-node-resolver/0.log"
May 28 18:19:32.123317 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:32.123285    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-46l5q_ec78263b-aee2-442b-8bc0-5f738c3d6832/node-ca/0.log"
May 28 18:19:32.937041 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:32.937008    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4b424c8_33a6915a-2203-4ece-92ff-235f6e7f826b/istio-proxy/0.log"
May 28 18:19:33.125026 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:33.124981    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-7tpzj_214a7f13-c84e-42d1-b854-a8cc63b9de71/istio-proxy/0.log"
May 28 18:19:33.741886 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:33.741861    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t9dpl_f1717a8a-a4ee-4b09-b3c2-b06dbf730baf/serve-healthcheck-canary/0.log"
May 28 18:19:34.281282 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:34.281250    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9fg6h_ae0d039a-6315-4200-a6a7-375dbc82f2c9/kube-rbac-proxy/0.log"
May 28 18:19:34.301425 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:34.301397    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9fg6h_ae0d039a-6315-4200-a6a7-375dbc82f2c9/exporter/0.log"
May 28 18:19:34.320864 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:34.320830    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9fg6h_ae0d039a-6315-4200-a6a7-375dbc82f2c9/extractor/0.log"
May 28 18:19:36.267440 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:36.267396    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29666520-5c5vb_f04bd314-2659-4689-b765-9b997e63a2bf/cleanup/0.log"
May 28 18:19:36.309272 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:36.309245    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29666535-z5xg2_7e795b25-3857-4af2-b00e-2d23e1e4e617/cleanup/0.log"
May 28 18:19:36.488508 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:36.488467    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65fbbfd459-2fsr8_0970116a-e87a-4db0-8dd2-d2eb5f64f236/manager/0.log"
May 28 18:19:36.556672 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:36.556640    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-k2299_e44f759f-b13e-4295-b282-658fff395205/postgres/0.log"
May 28 18:19:37.564793 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:37.564765    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-v62rz/perf-node-gather-daemonset-xslrl"
May 28 18:19:37.653668 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:37.653635    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-655c6bbdd5-jtvwd_48249e43-e627-467c-bc0f-76c2fd301a55/manager/0.log"
May 28 18:19:43.480550 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:43.480511    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-27ntt_91d2d37d-cc9b-4ebb-a1f4-d312ad1cef8f/kube-multus/0.log"
May 28 18:19:43.832099 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:43.832060    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k47r7_f454ed50-48e4-4202-88d3-1402e75de767/kube-multus-additional-cni-plugins/0.log"
May 28 18:19:43.852826 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:43.852798    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k47r7_f454ed50-48e4-4202-88d3-1402e75de767/egress-router-binary-copy/0.log"
May 28 18:19:43.879188 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:43.879156    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k47r7_f454ed50-48e4-4202-88d3-1402e75de767/cni-plugins/0.log"
May 28 18:19:43.898829 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:43.898793    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k47r7_f454ed50-48e4-4202-88d3-1402e75de767/bond-cni-plugin/0.log"
May 28 18:19:43.920065 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:43.920035    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k47r7_f454ed50-48e4-4202-88d3-1402e75de767/routeoverride-cni/0.log"
May 28 18:19:43.940437 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:43.940412    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k47r7_f454ed50-48e4-4202-88d3-1402e75de767/whereabouts-cni-bincopy/0.log"
May 28 18:19:43.960870 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:43.960839    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k47r7_f454ed50-48e4-4202-88d3-1402e75de767/whereabouts-cni/0.log"
May 28 18:19:44.095884 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:44.095800    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rh246_4693da93-c00a-4dd7-8dd0-fb3be0c73ad4/network-metrics-daemon/0.log"
May 28 18:19:44.118786 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:44.118754    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rh246_4693da93-c00a-4dd7-8dd0-fb3be0c73ad4/kube-rbac-proxy/0.log"
May 28 18:19:44.998598 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:44.998565    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fsvt7_b2eba6df-3e3e-46c8-b3c8-a1160503d901/ovn-controller/0.log"
May 28 18:19:45.028591 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:45.028559    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fsvt7_b2eba6df-3e3e-46c8-b3c8-a1160503d901/ovn-acl-logging/0.log"
May 28 18:19:45.047611 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:45.047583    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fsvt7_b2eba6df-3e3e-46c8-b3c8-a1160503d901/kube-rbac-proxy-node/0.log"
May 28 18:19:45.069053 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:45.069022    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fsvt7_b2eba6df-3e3e-46c8-b3c8-a1160503d901/kube-rbac-proxy-ovn-metrics/0.log"
May 28 18:19:45.087115 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:45.087068    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fsvt7_b2eba6df-3e3e-46c8-b3c8-a1160503d901/northd/0.log"
May 28 18:19:45.105654 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:45.105628    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fsvt7_b2eba6df-3e3e-46c8-b3c8-a1160503d901/nbdb/0.log"
May 28 18:19:45.125790 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:45.125763    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fsvt7_b2eba6df-3e3e-46c8-b3c8-a1160503d901/sbdb/0.log"
May 28 18:19:45.255832 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:45.255755    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fsvt7_b2eba6df-3e3e-46c8-b3c8-a1160503d901/ovnkube-controller/0.log"
May 28 18:19:46.987743 ip-10-0-135-41 kubenswrapper[2560]: I0528 18:19:46.987716    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-ch7tt_0eec8ff8-0b7b-4d4d-a787-463843df5ef7/network-check-target-container/0.log"