Jul 08 04:18:25.368481 ip-10-0-133-10 systemd[1]: Starting Kubernetes Kubelet... Jul 08 04:18:25.790477 ip-10-0-133-10 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 04:18:25.790477 ip-10-0-133-10 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 04:18:25.790477 ip-10-0-133-10 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 04:18:25.790477 ip-10-0-133-10 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 04:18:25.790477 ip-10-0-133-10 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 04:18:25.790477 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.697789 2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 04:18:25.790477 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701166 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 04:18:25.790477 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701176 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 04:18:25.790477 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701179 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 04:18:25.790477 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701182 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 04:18:25.790477 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701186 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 04:18:25.790477 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701189 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 04:18:25.790477 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701192 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 04:18:25.790477 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701195 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 04:18:25.765299 ip-10-0-133-10 systemd[1]: Started Kubernetes Kubelet. Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701198 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701201 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701204 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701206 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701209 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701211 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701214 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701217 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701219 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701234 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701251 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701254 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701256 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701260 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701264 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701268 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701273 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701276 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 04:18:25.791363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701288 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701293 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701298 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701301 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701304 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701307 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701310 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701313 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701316 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701319 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701321 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701324 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701326 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701329 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701331 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701333 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701336 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701339 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701341 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 04:18:25.793200 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701343 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701352 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701355 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701357 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701360 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701362 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701365 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701367 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701370 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701372 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701375 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701378 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701381 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701383 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701386 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701389 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701391 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701394 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701397 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701400 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 04:18:25.795325 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701402 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701405 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701407 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701410 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701414 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701416 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701419 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701421 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701424 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701426 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701429 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701431 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701433 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701436 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701439 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701441 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701444 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701446 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701449 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701451 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 04:18:25.797767 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701454 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701458 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701461 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701463 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701467 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701470 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701472 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701475 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701478 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701481 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701483 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701486 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701488 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.701491 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702141 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702148 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702150 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702154 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702158 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 04:18:25.801144 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702161 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702164 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702167 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702170 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702172 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702175 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702177 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702180 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702183 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702186 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702188 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702191 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702193 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702196 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702198 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702200 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702203 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702207 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702209 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702212 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 04:18:25.804175 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702214 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702217 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702219 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702222 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702225 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702227 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702230 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702232 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702235 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702251 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702254 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702257 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702260 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702262 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702265 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702267 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702270 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702273 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702275 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702278 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 04:18:25.805595 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702281 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702284 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702287 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702289 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702292 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702294 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702297 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702299 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702302 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702304 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702307 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702309 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702314 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702318 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702321 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702324 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702327 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702330 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702332 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 04:18:25.806647 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702335 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702337 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702340 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702343 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702345 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702348 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702351 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702353 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702355 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702358 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702361 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702364 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702367 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702369 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702372 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702381 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702384 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702387 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702389 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702392 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 04:18:25.807657 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702395 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702398 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702401 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702403 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702406 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702408 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702411 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702415 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702417 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702420 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702423 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702426 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702430 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702434 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702437 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702773 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702778 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702781 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702784 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702786 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 04:18:25.808494 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702789 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702791 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702794 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702796 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702800 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702803 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702806 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702809 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702812 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702815 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702817 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702820 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702822 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702825 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702828 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702830 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702833 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702836 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702838 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702841 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 04:18:25.809069 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702843 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702846 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702848 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702851 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702853 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702856 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702858 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702861 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702864 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702868 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702870 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702873 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702876 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702880 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702882 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702885 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702888 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702891 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702894 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 04:18:25.809652 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702896 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702900 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702903 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702905 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702908 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702910 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702913 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702916 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702919 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702921 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702928 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702930 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702933 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702937 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702939 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702943 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702945 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702948 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702950 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702953 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 04:18:25.810182 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702955 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702958 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702960 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702962 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702965 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702968 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702971 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702975 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702978 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702981 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702983 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702986 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702988 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702991 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702993 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702996 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.702998 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703001 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703003 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 04:18:25.810773 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703005 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703008 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703011 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703014 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703017 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703019 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703024 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703027 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703030 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703032 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703035 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703037 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703040 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703042 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703045 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703047 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703116 2583 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703122 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703127 2583 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703132 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703135 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 04:18:25.811303 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703139 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703142 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703146 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703149 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703152 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703155 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703159 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703162 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703165 2583 flags.go:64] FLAG: --cgroup-root="" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703168 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703171 2583 flags.go:64] FLAG: --client-ca-file="" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703174 2583 flags.go:64] FLAG: --cloud-provider="external" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703177 2583 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703182 2583 flags.go:64] FLAG: --cluster-domain="" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703186 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703189 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703192 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703196 2583 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703200 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703202 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703205 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703208 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703211 2583 flags.go:64] FLAG: --contention-profiling="false" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703214 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 04:18:25.811868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703218 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703221 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703224 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703228 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703231 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703234 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703250 2583 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703253 2583 flags.go:64] FLAG: --enable-server="true" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703256 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703262 2583 flags.go:64] FLAG: --event-burst="100" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703267 2583 flags.go:64] FLAG: --event-qps="50" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703271 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703276 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703280 2583 flags.go:64] FLAG: --eviction-hard="" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703284 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703287 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703291 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703294 2583 flags.go:64] FLAG: --eviction-soft="" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703296 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703299 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703302 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703305 2583 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703309 2583 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703312 2583 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703315 2583 flags.go:64] FLAG: --feature-gates="" Jul 08 04:18:25.812539 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703320 2583 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703323 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703326 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703328 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703332 2583 flags.go:64] FLAG: --healthz-port="10248" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703335 2583 flags.go:64] FLAG: --help="false" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703337 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-133-10.ec2.internal" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703340 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703343 2583 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703346 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703349 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703352 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703355 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703358 2583 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703360 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703363 2583 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703366 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703369 2583 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703372 2583 flags.go:64] FLAG: --kube-reserved="" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703374 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703377 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703380 2583 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703383 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703386 2583 flags.go:64] FLAG: --lock-file="" Jul 08 04:18:25.813568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703389 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703392 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703395 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703400 2583 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703403 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703407 2583 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703409 2583 flags.go:64] FLAG: --logging-format="text" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703412 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703417 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703419 2583 flags.go:64] FLAG: --manifest-url="" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703422 2583 flags.go:64] FLAG: --manifest-url-header="" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703426 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703429 2583 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703433 2583 flags.go:64] FLAG: --max-pods="110" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703436 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703439 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703442 2583 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703445 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703448 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703451 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703453 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703460 2583 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703463 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703466 2583 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 04:18:25.814931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703469 2583 flags.go:64] FLAG: --pod-cidr="" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703472 2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703477 2583 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703480 2583 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703483 2583 flags.go:64] FLAG: --pods-per-core="0" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703486 2583 flags.go:64] FLAG: --port="10250" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703488 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703492 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b7433325dc9ec492" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703495 2583 flags.go:64] FLAG: --qos-reserved="" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703498 2583 flags.go:64] FLAG: --read-only-port="10255" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703501 2583 flags.go:64] FLAG: --register-node="true" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703504 2583 flags.go:64] FLAG: --register-with-taints="" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703507 2583 flags.go:64] FLAG: --registry-burst="10" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703512 2583 flags.go:64] FLAG: --registry-qps="5" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703515 2583 flags.go:64] FLAG: --reserved-cpus="" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703517 2583 flags.go:64] FLAG: --reserved-memory="" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703521 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703525 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703527 2583 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703530 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703533 2583 flags.go:64] FLAG: --runonce="false" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703536 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703539 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703542 2583 flags.go:64] FLAG: --seccomp-default="false" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703545 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 04:18:25.815621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703548 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703551 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703554 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703557 2583 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703559 2583 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703562 2583 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703565 2583 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703568 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703570 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703573 2583 flags.go:64] FLAG: --system-cgroups="" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703576 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703581 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703583 2583 flags.go:64] FLAG: --tls-cert-file="" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703586 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703591 2583 flags.go:64] FLAG: --tls-min-version="" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703594 2583 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703596 2583 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703600 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703602 2583 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703605 2583 flags.go:64] FLAG: --v="2" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703610 2583 flags.go:64] FLAG: --version="false" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703614 2583 flags.go:64] FLAG: --vmodule="" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703618 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.703621 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703731 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 04:18:26.087881 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703734 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703738 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703741 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703744 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703746 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703749 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703753 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703757 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703760 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703763 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703765 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703768 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703770 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703773 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703775 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703778 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703781 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703783 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703786 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 04:18:26.109872 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703789 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703791 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703794 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703796 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703798 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703801 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703804 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703806 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703811 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703813 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703816 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703818 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703822 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703824 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703827 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703829 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703832 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703834 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703837 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703839 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 04:18:26.110544 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703842 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703844 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703847 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703850 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703852 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703854 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703857 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703859 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703862 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703864 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703868 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703872 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703874 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703877 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703879 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703882 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703885 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703887 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703890 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 04:18:26.111192 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703893 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703896 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703899 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703901 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703904 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703907 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703910 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703912 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703915 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703917 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703920 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703923 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703926 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703928 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703931 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703933 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703936 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703939 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703941 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 04:18:26.111802 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703944 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703946 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703949 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703952 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703954 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703957 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703959 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703962 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703964 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703967 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703969 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703971 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703974 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703977 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703981 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703983 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703986 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703989 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703992 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703995 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 04:18:26.112396 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.703997 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 04:18:26.112938 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.704579 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 04:18:26.112938 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.711236 2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 04:18:26.112938 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.711269 2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 04:18:26.112938 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.711897 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 04:18:26.112938 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.711907 2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 04:18:26.112938 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.711961 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 04:18:26.112938 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.711966 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 04:18:26.112938 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.711970 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 04:18:26.112938 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.711973 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 04:18:26.112938 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.711976 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 04:18:26.112938 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.711980 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 04:18:26.112938 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.711984 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 04:18:26.112938 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.711987 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 04:18:26.112938 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.711990 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.711994 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.711997 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.711999 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712002 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712005 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712008 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712010 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712013 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712016 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712018 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712021 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712026 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712030 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712032 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712035 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712037 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712042 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712046 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712049 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 04:18:26.113436 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712052 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712055 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712057 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712060 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712063 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712066 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712068 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712071 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712074 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712076 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712079 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712082 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712084 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712087 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712089 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712092 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712095 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712097 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712100 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712103 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 04:18:26.113984 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712106 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712109 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712111 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712114 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712118 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712121 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712123 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712126 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712128 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712132 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712135 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712137 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712139 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712142 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712144 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712147 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712149 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712152 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712155 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 04:18:26.114582 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712157 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712161 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712164 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712167 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712170 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712172 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712175 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712178 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712180 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712183 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712185 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712188 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712191 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712193 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712196 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712198 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712201 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712204 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712207 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 04:18:26.115093 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712210 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 04:18:26.115667 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712213 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 04:18:26.115667 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712215 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 04:18:26.115667 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712219 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 04:18:26.115667 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712221 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 04:18:26.115667 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712224 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 04:18:26.115667 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712227 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 04:18:26.115667 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712229 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 04:18:26.115667 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712232 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 04:18:26.115667 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712234 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 04:18:26.115667 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712253 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 04:18:26.115667 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712258 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 04:18:26.115667 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712262 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 04:18:26.115667 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.712268 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 04:18:26.115667 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712370 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 04:18:26.115667 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712374 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712377 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712380 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712382 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712385 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712387 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712390 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712392 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712395 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712397 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712399 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712402 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712405 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712407 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712410 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712414 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712416 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712419 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712422 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 04:18:26.116091 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712424 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712428 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712432 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712435 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712437 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712440 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712443 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712446 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712448 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712451 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712453 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712456 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712458 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712461 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712463 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712466 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712469 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712471 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712473 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712476 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 04:18:26.116664 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712478 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712481 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712483 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712486 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712488 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712491 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712493 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712496 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712499 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712502 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712504 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712507 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712509 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712512 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712516 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712518 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712521 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712524 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712526 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712529 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 04:18:26.117204 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712532 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712534 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712537 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712539 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712542 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712544 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712546 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712549 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712551 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712554 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712556 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712559 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712561 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712564 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712566 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712569 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712571 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712575 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712578 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 04:18:26.120134 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712581 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712585 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712588 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712590 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712593 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712595 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712598 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712601 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712604 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712607 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712610 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712614 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712617 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712619 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712622 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712625 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712628 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712630 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712633 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 04:18:26.120677 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:25.712636 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 04:18:26.121230 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.712640 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 04:18:26.121230 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.712735 2583 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 04:18:26.121230 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.715508 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 04:18:26.121230 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.716723 2583 server.go:1019] "Starting client certificate rotation" Jul 08 04:18:26.121230 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.716823 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 04:18:26.121230 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.717435 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 04:18:26.121230 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.721650 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 04:18:26.121230 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.722702 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 04:18:26.121230 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.733651 2583 log.go:25] "Validated CRI v1 runtime API" Jul 08 04:18:26.121230 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.735763 2583 log.go:25] "Validated CRI v1 image API" Jul 08 04:18:26.121230 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.737162 2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 04:18:26.121230 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.739196 2583 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9aafea25-f927-4439-ad39-7e9881cd35b6:/dev/nvme0n1p3 ccdab015-7ff6-4420-b5be-844275f85ac8:/dev/nvme0n1p4] Jul 08 04:18:26.121594 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.739210 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 04:18:26.121706 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.744327 2583 manager.go:216] Machine: {Timestamp:2026-07-08 04:18:25.743087389 +0000 UTC m=+0.274966060 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097430 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2576c7caebb98e88593287a058ca65 SystemUUID:ec2576c7-caeb-b98e-8859-3287a058ca65 BootID:3ffc26a9-ff62-4852-bbef-33554d405318 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e7:d0:be:48:95 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e7:d0:be:48:95 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:70:fc:82:54:af Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 04:18:26.121706 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.744443 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 04:18:26.121706 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.744517 2583 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 04:18:26.121706 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.745843 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 04:18:26.121706 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.747194 2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 04:18:26.121706 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.747216 2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-10.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 04:18:26.121706 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.747885 2583 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.747895 2583 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.747903 2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.748465 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.749676 2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.753551 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.753665 2583 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.753691 2583 kubelet.go:486] "Attempting to sync node with API server" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.753700 2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.753714 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.753722 2583 kubelet.go:398] "Adding apiserver pod source" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.753731 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.754052 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.754067 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.757348 2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.759221 2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.759267 2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.760352 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.760374 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.760381 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.760386 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.760394 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.760399 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.760407 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.760412 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.760420 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.760428 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.760438 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.760443 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.761057 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.761065 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.764335 2583 server.go:1268] "Started kubelet" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.764423 2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.764474 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.764542 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.765285 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.765308 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.765410 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.766086 2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 04:18:26.122424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.766544 2583 server.go:310] "Adding debug handlers to kubelet server" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.768648 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586db90dc2d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.764310061 +0000 UTC m=+0.296188730,LastTimestamp:2026-07-08 04:18:25.764310061 +0000 UTC m=+0.296188730,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.769718 2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.771570 2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.771575 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.771658 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-10.ec2.internal\" not found" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.772420 2583 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.772439 2583 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.772564 2583 reconstruct.go:97] "Volume reconstruction finished" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.772571 2583 reconciler.go:29] "Reconciler: start to sync state" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.773394 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.773774 2583 factory.go:55] Registering systemd factory Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.773776 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.773787 2583 factory.go:223] Registration of the systemd container factory successfully Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.774029 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4jlq8" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.774088 2583 factory.go:153] Registering CRI-O factory Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.774099 2583 factory.go:223] Registration of the crio container factory successfully Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.774148 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.774169 2583 factory.go:103] Registering Raw factory Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.774183 2583 manager.go:1196] Started watching for new ooms in manager Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.774594 2583 manager.go:349] Starting recovery of all containers Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.784432 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.786691 2583 manager.go:354] Recovery completed Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.791426 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.793254 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.794049 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.794059 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.794658 2583 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.794669 2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.794679 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.794684 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.795408 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:26.123566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.795429 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:28.431473 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.795441 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:28.431473 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.797042 2583 policy_none.go:49] "None policy: Start" Jul 08 04:18:28.431473 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.797061 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 04:18:28.431473 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.797077 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 04:18:28.431473 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.799101 2583 policy_none.go:47] "Start" Jul 08 04:18:28.431473 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.803100 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd5658ce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794029774 +0000 UTC m=+0.325908444,LastTimestamp:2026-07-08 04:18:25.794029774 +0000 UTC m=+0.325908444,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.431473 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.807550 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd56b7ed default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794054125 +0000 UTC m=+0.325932795,LastTimestamp:2026-07-08 04:18:25.794054125 +0000 UTC m=+0.325932795,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.431473 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.812837 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd56de1e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794063902 +0000 UTC m=+0.325942571,LastTimestamp:2026-07-08 04:18:25.794063902 +0000 UTC m=+0.325942571,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.431473 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.816173 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd5658ce\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd5658ce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794029774 +0000 UTC m=+0.325908444,LastTimestamp:2026-07-08 04:18:25.795419995 +0000 UTC m=+0.327298669,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.823455 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd56b7ed\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd56b7ed default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794054125 +0000 UTC m=+0.325932795,LastTimestamp:2026-07-08 04:18:25.795434485 +0000 UTC m=+0.327313158,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.825862 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd56de1e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd56de1e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794063902 +0000 UTC m=+0.325942571,LastTimestamp:2026-07-08 04:18:25.795446808 +0000 UTC m=+0.327325479,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.843750 2583 manager.go:337] "Starting Device Plugin manager" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.843781 2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.843791 2583 server.go:87] "Starting device plugin registration server" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.844020 2583 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.844041 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.844123 2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.844193 2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.844202 2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.844645 2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.844698 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-10.ec2.internal\" not found" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.847545 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586e06c54b2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.845802162 +0000 UTC m=+0.377680823,LastTimestamp:2026-07-08 04:18:25.845802162 +0000 UTC m=+0.377680823,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.944705 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.945693 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.945717 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.945728 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:28.432118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.945755 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.432745 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.954881 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd5658ce\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd5658ce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794029774 +0000 UTC m=+0.325908444,LastTimestamp:2026-07-08 04:18:25.945704081 +0000 UTC m=+0.477582751,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.432745 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.963039 2583 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.432745 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.963145 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd56b7ed\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd56b7ed default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794054125 +0000 UTC m=+0.325932795,LastTimestamp:2026-07-08 04:18:25.945722068 +0000 UTC m=+0.477600738,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.432745 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.971389 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 04:18:28.432745 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.972370 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd56de1e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd56de1e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794063902 +0000 UTC m=+0.325942571,LastTimestamp:2026-07-08 04:18:25.945732434 +0000 UTC m=+0.477611104,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.432745 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.972685 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 04:18:28.432745 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.972717 2583 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 04:18:28.432745 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:25.972737 2583 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 04:18:28.432745 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.972766 2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 04:18:28.432745 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.981116 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Jul 08 04:18:28.432745 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:25.997808 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms" Jul 08 04:18:28.432745 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.073139 2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal"] Jul 08 04:18:28.432745 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.073230 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.074684 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/26ba4c92c69564f7ea9cda441d8bf34b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal\" (UID: \"26ba4c92c69564f7ea9cda441d8bf34b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.074716 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26ba4c92c69564f7ea9cda441d8bf34b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal\" (UID: \"26ba4c92c69564f7ea9cda441d8bf34b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.075008 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.075035 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.075045 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.077331 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.077428 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.077457 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.078061 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.078090 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.078101 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.078064 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.078157 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.078167 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.080278 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.080300 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.080954 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.080982 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.080992 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:28.433251 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.083097 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd5658ce\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd5658ce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794029774 +0000 UTC m=+0.325908444,LastTimestamp:2026-07-08 04:18:26.075020919 +0000 UTC m=+0.606899589,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.433824 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.095381 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd56b7ed\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd56b7ed default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794054125 +0000 UTC m=+0.325932795,LastTimestamp:2026-07-08 04:18:26.075039972 +0000 UTC m=+0.606918641,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.433824 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.100750 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-10.ec2.internal\" not found" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.433824 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.103309 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd56de1e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd56de1e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794063902 +0000 UTC m=+0.325942571,LastTimestamp:2026-07-08 04:18:26.075048572 +0000 UTC m=+0.606927241,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.433824 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.106576 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-10.ec2.internal\" not found" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.433824 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.111056 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd5658ce\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd5658ce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794029774 +0000 UTC m=+0.325908444,LastTimestamp:2026-07-08 04:18:26.078079539 +0000 UTC m=+0.609958215,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.433824 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.119607 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd56b7ed\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd56b7ed default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794054125 +0000 UTC m=+0.325932795,LastTimestamp:2026-07-08 04:18:26.078095501 +0000 UTC m=+0.609974171,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.434086 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.128345 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd56de1e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd56de1e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794063902 +0000 UTC m=+0.325942571,LastTimestamp:2026-07-08 04:18:26.078106534 +0000 UTC m=+0.609985204,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.434086 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.135786 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd5658ce\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd5658ce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794029774 +0000 UTC m=+0.325908444,LastTimestamp:2026-07-08 04:18:26.078146234 +0000 UTC m=+0.610024904,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.434086 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.145031 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd56b7ed\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd56b7ed default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794054125 +0000 UTC m=+0.325932795,LastTimestamp:2026-07-08 04:18:26.078162762 +0000 UTC m=+0.610041432,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.434086 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.156713 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd56de1e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd56de1e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794063902 +0000 UTC m=+0.325942571,LastTimestamp:2026-07-08 04:18:26.078171342 +0000 UTC m=+0.610050015,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.434086 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.163754 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:28.434086 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.164589 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:28.434086 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.164614 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:28.434086 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.164623 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:28.434086 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.164644 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.434391 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.170258 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd5658ce\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd5658ce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794029774 +0000 UTC m=+0.325908444,LastTimestamp:2026-07-08 04:18:26.080965626 +0000 UTC m=+0.612844298,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.434391 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.175372 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26ba4c92c69564f7ea9cda441d8bf34b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal\" (UID: \"26ba4c92c69564f7ea9cda441d8bf34b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.434391 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.175395 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/114dd0440d9cd8b63ab11e6e15492e28-config\") pod \"kube-apiserver-proxy-ip-10-0-133-10.ec2.internal\" (UID: \"114dd0440d9cd8b63ab11e6e15492e28\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.434391 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.175411 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/26ba4c92c69564f7ea9cda441d8bf34b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal\" (UID: \"26ba4c92c69564f7ea9cda441d8bf34b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.434391 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.175440 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/26ba4c92c69564f7ea9cda441d8bf34b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal\" (UID: \"26ba4c92c69564f7ea9cda441d8bf34b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.434391 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:26.175477 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26ba4c92c69564f7ea9cda441d8bf34b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal\" (UID: \"26ba4c92c69564f7ea9cda441d8bf34b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.434391 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.177956 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd56b7ed\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd56b7ed default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794054125 +0000 UTC m=+0.325932795,LastTimestamp:2026-07-08 04:18:26.080987368 +0000 UTC m=+0.612866038,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.434391 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.178064 2583 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.434695 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.182847 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd56de1e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd56de1e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794063902 +0000 UTC m=+0.325942571,LastTimestamp:2026-07-08 04:18:26.080996082 +0000 UTC m=+0.612874751,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.434695 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.187487 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd5658ce\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd5658ce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794029774 +0000 UTC m=+0.325908444,LastTimestamp:2026-07-08 04:18:26.16460163 +0000 UTC m=+0.696480299,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.434695 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:26.195084 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-10.ec2.internal.18c03586dd56b7ed\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-10.ec2.internal.18c03586dd56b7ed default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-10.ec2.internal,UID:ip-10-0-133-10.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-10.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:25.794054125 +0000 UTC m=+0.325932795,LastTimestamp:2026-07-08 04:18:26.164618893 +0000 UTC m=+0.696497563,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:28.434695 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:28.370935 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/114dd0440d9cd8b63ab11e6e15492e28-config\") pod \"kube-apiserver-proxy-ip-10-0-133-10.ec2.internal\" (UID: \"114dd0440d9cd8b63ab11e6e15492e28\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.434695 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:28.370890 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/114dd0440d9cd8b63ab11e6e15492e28-config\") pod \"kube-apiserver-proxy-ip-10-0-133-10.ec2.internal\" (UID: \"114dd0440d9cd8b63ab11e6e15492e28\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.714904 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:28.498291 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.714904 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:28.501381 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="800ms" Jul 08 04:18:28.714904 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:28.505224 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.714904 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:28.673613 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:28.714904 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:28.674426 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:28.714904 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:28.674455 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:28.714904 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:28.674465 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:28.714904 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:28.674491 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.714904 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:28.692283 2583 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:28.955635 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:28.771929 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 04:18:28.955635 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:28.870433 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:18:29.069109 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:29.004876 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 04:18:29.069109 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:29.043956 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:29.069109 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:29.045505 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:29.069109 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:29.045530 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:29.069109 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:29.045540 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:29.301877 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:29.301068 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 04:18:29.312122 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:29.311365 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="1.6s" Jul 08 04:18:29.416743 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:29.325637 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Jul 08 04:18:29.528033 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:29.492683 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:29.528033 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:29.493611 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:29.528033 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:29.493642 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:29.528033 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:29.493651 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:29.528033 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:29.493675 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:29.528033 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:29.511000 2583 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:29.898986 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:29.869489 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:18:30.410140 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:30.410109 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 04:18:30.419861 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:30.419817 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod114dd0440d9cd8b63ab11e6e15492e28.slice/crio-c2be2060bd6d02739f6616f1b9d06505764f013487c5fe7c05a39f4f73ee51c0 WatchSource:0}: Error finding container c2be2060bd6d02739f6616f1b9d06505764f013487c5fe7c05a39f4f73ee51c0: Status 404 returned error can't find the container with id c2be2060bd6d02739f6616f1b9d06505764f013487c5fe7c05a39f4f73ee51c0 Jul 08 04:18:30.420337 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:30.420302 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26ba4c92c69564f7ea9cda441d8bf34b.slice/crio-a4e8dcd5ea7a5f0a45b03112a37fdf2cd60e77a20ed0565a1931a224d13db124 WatchSource:0}: Error finding container a4e8dcd5ea7a5f0a45b03112a37fdf2cd60e77a20ed0565a1931a224d13db124: Status 404 returned error can't find the container with id a4e8dcd5ea7a5f0a45b03112a37fdf2cd60e77a20ed0565a1931a224d13db124 Jul 08 04:18:30.424915 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:30.424901 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 04:18:30.434827 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:30.434759 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal.18c03587f15f4695 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal,UID:26ba4c92c69564f7ea9cda441d8bf34b,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\",Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:30.425126549 +0000 UTC m=+2.861901506,LastTimestamp:2026-07-08 04:18:30.425126549 +0000 UTC m=+2.861901506,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:30.444359 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:30.444270 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-133-10.ec2.internal.18c03587f160140d kube-system 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-133-10.ec2.internal,UID:114dd0440d9cd8b63ab11e6e15492e28,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d6df0cb6282bc5ba571a8de322fe598afdf22e817f5a7dc92db3274dbdb9a1\",Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:30.425179149 +0000 UTC m=+2.861954107,LastTimestamp:2026-07-08 04:18:30.425179149 +0000 UTC m=+2.861954107,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:30.869615 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:30.869583 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:18:30.922377 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:30.922329 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="3.2s" Jul 08 04:18:31.074569 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:31.074512 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" event={"ID":"26ba4c92c69564f7ea9cda441d8bf34b","Type":"ContainerStarted","Data":"a4e8dcd5ea7a5f0a45b03112a37fdf2cd60e77a20ed0565a1931a224d13db124"} Jul 08 04:18:31.075946 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:31.075911 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" event={"ID":"114dd0440d9cd8b63ab11e6e15492e28","Type":"ContainerStarted","Data":"c2be2060bd6d02739f6616f1b9d06505764f013487c5fe7c05a39f4f73ee51c0"} Jul 08 04:18:31.111335 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:31.111309 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:31.112718 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:31.112514 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:31.112718 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:31.112558 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:31.112718 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:31.112575 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:31.112718 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:31.112610 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:31.131061 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:31.130999 2583 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:31.171411 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:31.171388 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:31.172689 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:31.172669 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:31.172781 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:31.172702 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:31.172781 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:31.172716 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:31.511986 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:31.511955 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 04:18:31.753313 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:31.753278 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Jul 08 04:18:31.753313 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:31.753281 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 04:18:31.862893 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:31.862821 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:18:32.404007 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:32.403913 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal.18c0358866beb879 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal,UID:26ba4c92c69564f7ea9cda441d8bf34b,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\" in 1.969s (1.969s including waiting). Image size: 303912755 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:32.394315897 +0000 UTC m=+4.831090854,LastTimestamp:2026-07-08 04:18:32.394315897 +0000 UTC m=+4.831090854,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:32.443057 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:32.442958 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-133-10.ec2.internal.18c035886918488f kube-system 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-133-10.ec2.internal,UID:114dd0440d9cd8b63ab11e6e15492e28,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32d6df0cb6282bc5ba571a8de322fe598afdf22e817f5a7dc92db3274dbdb9a1\" in 2.008s (2.008s including waiting). Image size: 342879910 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:32.433739919 +0000 UTC m=+4.870514875,LastTimestamp:2026-07-08 04:18:32.433739919 +0000 UTC m=+4.870514875,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:32.503156 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:32.503090 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-133-10.ec2.internal.18c035886cb38572 kube-system 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-133-10.ec2.internal,UID:114dd0440d9cd8b63ab11e6e15492e28,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Created,Message:Created container: haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:32.494245234 +0000 UTC m=+4.931020203,LastTimestamp:2026-07-08 04:18:32.494245234 +0000 UTC m=+4.931020203,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:32.510730 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:32.510649 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-133-10.ec2.internal.18c035886d1fb903 kube-system 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-133-10.ec2.internal,UID:114dd0440d9cd8b63ab11e6e15492e28,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Started,Message:Started container haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:32.501336323 +0000 UTC m=+4.938111280,LastTimestamp:2026-07-08 04:18:32.501336323 +0000 UTC m=+4.938111280,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:32.654251 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:32.654064 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal.18c0358875ca0c8a openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal,UID:26ba4c92c69564f7ea9cda441d8bf34b,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:32.646716554 +0000 UTC m=+5.083491524,LastTimestamp:2026-07-08 04:18:32.646716554 +0000 UTC m=+5.083491524,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:32.662734 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:32.662670 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal.18c0358876457a4a openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal,UID:26ba4c92c69564f7ea9cda441d8bf34b,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:32.654805578 +0000 UTC m=+5.091580553,LastTimestamp:2026-07-08 04:18:32.654805578 +0000 UTC m=+5.091580553,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:32.870666 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:32.870642 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:18:33.081262 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:33.081219 2583 generic.go:356] "Generic (PLEG): container finished" podID="26ba4c92c69564f7ea9cda441d8bf34b" containerID="fc0bfb89f7754d257f4a6c45d9ff08f82b70c0425b3b397a32512f84d7041ac1" exitCode=0 Jul 08 04:18:33.081398 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:33.081285 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" event={"ID":"26ba4c92c69564f7ea9cda441d8bf34b","Type":"ContainerDied","Data":"fc0bfb89f7754d257f4a6c45d9ff08f82b70c0425b3b397a32512f84d7041ac1"} Jul 08 04:18:33.081398 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:33.081298 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:33.082227 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:33.082210 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:33.082320 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:33.082233 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:33.082320 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:33.082247 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:33.082499 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:33.082485 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-10.ec2.internal\" not found" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:33.082649 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:33.082629 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" event={"ID":"114dd0440d9cd8b63ab11e6e15492e28","Type":"ContainerStarted","Data":"2f1e7a6d7eb046debd0712b3bca56bf7c8ee5ac6845c3770b3660e7520480c13"} Jul 08 04:18:33.082699 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:33.082666 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:33.083339 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:33.083322 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:33.083416 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:33.083371 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:33.083416 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:33.083383 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:33.083527 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:33.083516 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-10.ec2.internal\" not found" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:33.093073 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:33.093009 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal.18c035888fe1702a openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal,UID:26ba4c92c69564f7ea9cda441d8bf34b,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-rbac-proxy-crio},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:33.084457002 +0000 UTC m=+5.521231958,LastTimestamp:2026-07-08 04:18:33.084457002 +0000 UTC m=+5.521231958,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:33.180112 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:33.180044 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal.18c0358895178d34 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal,UID:26ba4c92c69564f7ea9cda441d8bf34b,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-rbac-proxy-crio},},Reason:Created,Message:Created container: kube-rbac-proxy-crio,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:33.17188946 +0000 UTC m=+5.608664417,LastTimestamp:2026-07-08 04:18:33.17188946 +0000 UTC m=+5.608664417,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:33.188873 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:33.188802 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal.18c0358895918a76 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal,UID:26ba4c92c69564f7ea9cda441d8bf34b,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-rbac-proxy-crio},},Reason:Started,Message:Started container kube-rbac-proxy-crio,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:33.17988415 +0000 UTC m=+5.616659130,LastTimestamp:2026-07-08 04:18:33.17988415 +0000 UTC m=+5.616659130,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:33.639567 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:33.639537 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 04:18:33.869530 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:33.869497 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:18:34.085208 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.085180 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/0.log" Jul 08 04:18:34.085538 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.085516 2583 generic.go:356] "Generic (PLEG): container finished" podID="26ba4c92c69564f7ea9cda441d8bf34b" containerID="b66fe276dfea81ba4ddbe2edfe99cd5851cf280563859f1cd4f64f8fbd7fed1f" exitCode=1 Jul 08 04:18:34.085617 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.085549 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" event={"ID":"26ba4c92c69564f7ea9cda441d8bf34b","Type":"ContainerDied","Data":"b66fe276dfea81ba4ddbe2edfe99cd5851cf280563859f1cd4f64f8fbd7fed1f"} Jul 08 04:18:34.085617 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.085602 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:34.085617 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.085610 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:34.086402 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.086383 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:34.086518 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.086417 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:34.086518 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.086432 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:34.086518 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.086383 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:34.086518 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.086490 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:34.086518 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.086501 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:34.086739 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:34.086650 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-10.ec2.internal\" not found" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:34.086739 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:34.086693 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-10.ec2.internal\" not found" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:34.086739 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.086712 2583 scope.go:117] "RemoveContainer" containerID="b66fe276dfea81ba4ddbe2edfe99cd5851cf280563859f1cd4f64f8fbd7fed1f" Jul 08 04:18:34.099457 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:34.099370 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal.18c035888fe1702a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal.18c035888fe1702a openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal,UID:26ba4c92c69564f7ea9cda441d8bf34b,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-rbac-proxy-crio},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:33.084457002 +0000 UTC m=+5.521231958,LastTimestamp:2026-07-08 04:18:34.089587959 +0000 UTC m=+6.526362916,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:34.132723 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:34.132609 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="6.4s" Jul 08 04:18:34.179055 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:34.178967 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal.18c0358895178d34\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal.18c0358895178d34 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal,UID:26ba4c92c69564f7ea9cda441d8bf34b,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-rbac-proxy-crio},},Reason:Created,Message:Created container: kube-rbac-proxy-crio,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:33.17188946 +0000 UTC m=+5.608664417,LastTimestamp:2026-07-08 04:18:34.170430179 +0000 UTC m=+6.607205137,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:34.193880 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:34.193789 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal.18c0358895918a76\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal.18c0358895918a76 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal,UID:26ba4c92c69564f7ea9cda441d8bf34b,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-rbac-proxy-crio},},Reason:Started,Message:Started container kube-rbac-proxy-crio,Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:33.17988415 +0000 UTC m=+5.616659130,LastTimestamp:2026-07-08 04:18:34.18458604 +0000 UTC m=+6.621361021,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:34.331750 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.331710 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:34.332834 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.332809 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:34.332964 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.332848 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:34.332964 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.332860 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:34.332964 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.332887 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:34.349052 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:34.349004 2583 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:34.869195 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:34.869168 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:18:35.088612 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:35.088583 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:18:35.089050 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:35.088981 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/0.log" Jul 08 04:18:35.089296 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:35.089276 2583 generic.go:356] "Generic (PLEG): container finished" podID="26ba4c92c69564f7ea9cda441d8bf34b" containerID="4d301963bf4fde2d84ad604254a23a2af77e392652d03bf980ac466fde70fa39" exitCode=1 Jul 08 04:18:35.089332 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:35.089310 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" event={"ID":"26ba4c92c69564f7ea9cda441d8bf34b","Type":"ContainerDied","Data":"4d301963bf4fde2d84ad604254a23a2af77e392652d03bf980ac466fde70fa39"} Jul 08 04:18:35.089387 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:35.089369 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:35.089387 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:35.089376 2583 scope.go:117] "RemoveContainer" containerID="b66fe276dfea81ba4ddbe2edfe99cd5851cf280563859f1cd4f64f8fbd7fed1f" Jul 08 04:18:35.090828 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:35.090810 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:35.090894 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:35.090839 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:35.090894 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:35.090848 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:35.091096 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:35.091081 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-10.ec2.internal\" not found" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:35.091152 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:35.091141 2583 scope.go:117] "RemoveContainer" containerID="4d301963bf4fde2d84ad604254a23a2af77e392652d03bf980ac466fde70fa39" Jul 08 04:18:35.091277 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:35.091261 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-rbac-proxy-crio\" with CrashLoopBackOff: \"back-off 10s restarting failed container=kube-rbac-proxy-crio pod=kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_openshift-machine-config-operator(26ba4c92c69564f7ea9cda441d8bf34b)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" podUID="26ba4c92c69564f7ea9cda441d8bf34b" Jul 08 04:18:35.102036 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:35.101964 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal.18c03589077e56d3 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal,UID:26ba4c92c69564f7ea9cda441d8bf34b,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-rbac-proxy-crio},},Reason:BackOff,Message:Back-off restarting failed container kube-rbac-proxy-crio in pod kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_openshift-machine-config-operator(26ba4c92c69564f7ea9cda441d8bf34b),Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:35.091228371 +0000 UTC m=+7.528003344,LastTimestamp:2026-07-08 04:18:35.091228371 +0000 UTC m=+7.528003344,Count:1,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:35.861136 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:35.861108 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:35.862540 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:35.862523 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:35.862621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:35.862558 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:35.862621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:35.862571 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:35.869519 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:35.869499 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:18:36.077467 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:36.077433 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 04:18:36.091776 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:36.091715 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:18:36.092130 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:36.092116 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:36.092920 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:36.092902 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:36.092992 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:36.092933 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:36.092992 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:36.092943 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:36.093164 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:36.093152 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-10.ec2.internal\" not found" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:36.093206 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:36.093197 2583 scope.go:117] "RemoveContainer" containerID="4d301963bf4fde2d84ad604254a23a2af77e392652d03bf980ac466fde70fa39" Jul 08 04:18:36.093325 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:36.093309 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-rbac-proxy-crio\" with CrashLoopBackOff: \"back-off 10s restarting failed container=kube-rbac-proxy-crio pod=kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_openshift-machine-config-operator(26ba4c92c69564f7ea9cda441d8bf34b)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" podUID="26ba4c92c69564f7ea9cda441d8bf34b" Jul 08 04:18:36.102489 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:36.102415 2583 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal.18c03589077e56d3\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal.18c03589077e56d3 openshift-machine-config-operator 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal,UID:26ba4c92c69564f7ea9cda441d8bf34b,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{kube-rbac-proxy-crio},},Reason:BackOff,Message:Back-off restarting failed container kube-rbac-proxy-crio in pod kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_openshift-machine-config-operator(26ba4c92c69564f7ea9cda441d8bf34b),Source:EventSource{Component:kubelet,Host:ip-10-0-133-10.ec2.internal,},FirstTimestamp:2026-07-08 04:18:35.091228371 +0000 UTC m=+7.528003344,LastTimestamp:2026-07-08 04:18:36.093278512 +0000 UTC m=+8.530053469,Count:2,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-10.ec2.internal,}" Jul 08 04:18:36.202935 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:36.202908 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 04:18:36.867830 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:36.867799 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:18:37.870417 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:37.870385 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:18:37.940015 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:37.939983 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-10.ec2.internal\" not found" Jul 08 04:18:37.981252 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:37.981220 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Jul 08 04:18:38.869085 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:38.869057 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:18:39.869445 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:39.869416 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:18:40.543291 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:40.543258 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s" Jul 08 04:18:40.749805 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:40.749758 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:40.750790 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:40.750772 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:40.750898 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:40.750803 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:40.750898 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:40.750812 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:40.750898 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:40.750838 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:40.767811 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:40.767789 2583 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-133-10.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:40.871715 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:40.871654 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:18:41.868476 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:41.868445 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:18:42.870274 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:42.870242 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-10.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 04:18:43.097563 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:43.097533 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4jlq8" Jul 08 04:18:43.811849 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:43.811813 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 04:18:43.876413 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:43.876394 2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:43.892614 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:43.892596 2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:43.953109 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:43.953094 2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:44.103478 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:44.103407 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 04:13:43 +0000 UTC" deadline="2028-02-03 17:58:20.323624636 +0000 UTC" Jul 08 04:18:44.103478 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:44.103431 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13813h39m36.220195948s" Jul 08 04:18:44.228803 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:44.228783 2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:44.228915 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:44.228817 2583 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:44.266957 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:44.266935 2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:44.283441 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:44.283425 2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:44.339216 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:44.339196 2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:44.616291 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:44.616268 2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:44.616291 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:44.616293 2583 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:44.866715 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:44.866656 2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:44.882689 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:44.882669 2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:44.940550 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:44.940529 2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:45.205984 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:45.205963 2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:45.205984 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:45.205984 2583 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:45.310287 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:45.310266 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 04:18:45.345896 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:45.345876 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 04:18:45.464561 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:45.464508 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:45.465586 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:45.465572 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:45.465635 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:45.465599 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:45.465635 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:45.465608 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:45.859676 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:45.859616 2583 apiserver.go:52] "Watching apiserver" Jul 08 04:18:45.861888 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:45.861874 2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 04:18:45.861993 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:45.861975 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=[] Jul 08 04:18:45.868851 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:45.868837 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 04:18:46.061218 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:46.061194 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 04:18:46.503088 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:46.503060 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 04:18:47.200020 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.199993 2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:47.215834 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.215821 2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:47.276007 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.275991 2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:47.533236 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.533183 2583 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:47.533236 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:47.533209 2583 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-133-10.ec2.internal" not found Jul 08 04:18:47.549047 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:47.549028 2583 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-10.ec2.internal\" not found" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:47.768339 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.768282 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 04:18:47.769324 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.769308 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientMemory" Jul 08 04:18:47.769401 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.769358 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 04:18:47.769401 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.769372 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeHasSufficientPID" Jul 08 04:18:47.769401 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.769400 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:47.775463 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.775448 2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-10.ec2.internal" Jul 08 04:18:47.775500 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:47.775470 2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-10.ec2.internal\": node \"ip-10-0-133-10.ec2.internal\" not found" Jul 08 04:18:47.802542 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.802472 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-f545t"] Jul 08 04:18:47.804617 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.804602 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-58pdq"] Jul 08 04:18:47.804748 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.804734 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f545t" Jul 08 04:18:47.806045 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.806023 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-58pdq" Jul 08 04:18:47.807258 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.807214 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 04:18:47.807258 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.807256 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 04:18:47.807460 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.807414 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 04:18:47.807630 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.807617 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6rfhd\"" Jul 08 04:18:47.807710 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.807694 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 04:18:47.808530 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.808513 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fgc29\"" Jul 08 04:18:47.808614 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.808581 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 04:18:47.808700 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.808685 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 04:18:47.808750 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.808737 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 04:18:47.819279 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.819262 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-4rws5"] Jul 08 04:18:47.820970 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.820953 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-s4rvj"] Jul 08 04:18:47.821120 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.821104 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4rws5" Jul 08 04:18:47.823298 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.823276 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:47.824568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.824305 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 04:18:47.824568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.824317 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 04:18:47.824568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.824455 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 04:18:47.824843 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.824635 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-k254b\"" Jul 08 04:18:47.826464 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.825958 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wm6b8\"" Jul 08 04:18:47.826464 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.826284 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 04:18:47.827826 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.826518 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 04:18:47.866772 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.866755 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" Jul 08 04:18:47.878454 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.878433 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 04:18:47.896858 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.896839 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 04:18:47.902443 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.902424 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 04:18:47.902520 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.902492 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" Jul 08 04:18:47.902612 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.902596 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal"] Jul 08 04:18:47.902901 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.902890 2583 scope.go:117] "RemoveContainer" containerID="4d301963bf4fde2d84ad604254a23a2af77e392652d03bf980ac466fde70fa39" Jul 08 04:18:47.905965 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.905946 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-slm66"] Jul 08 04:18:47.907887 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.907871 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-slm66" Jul 08 04:18:47.910584 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.910566 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 04:18:47.910697 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.910566 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 04:18:47.910697 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.910619 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pc7zj\"" Jul 08 04:18:47.912640 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.912626 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 04:18:47.913664 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.913640 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-m5qfn"] Jul 08 04:18:47.915749 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.915730 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal"] Jul 08 04:18:47.916013 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.915940 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m5qfn" Jul 08 04:18:47.918943 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.918910 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 04:18:47.919104 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.919091 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kgj56\"" Jul 08 04:18:47.920712 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.920696 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 04:18:47.923704 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.923681 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vmkc7" Jul 08 04:18:47.932883 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.932008 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vmkc7" Jul 08 04:18:47.967453 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.967414 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-10.ec2.internal" podStartSLOduration=0.967401942 podStartE2EDuration="967.401942ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:18:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:47.967331827 +0000 UTC m=+20.404106800" watchObservedRunningTime="2026-07-08 04:18:47.967401942 +0000 UTC m=+20.404176921" Jul 08 04:18:47.972177 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972157 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbn4t\" (UniqueName: \"kubernetes.io/projected/5331e3cb-cabb-4cd9-b7ca-5977c7acaab7-kube-api-access-fbn4t\") pod \"node-ca-4rws5\" (UID: \"5331e3cb-cabb-4cd9-b7ca-5977c7acaab7\") " pod="openshift-image-registry/node-ca-4rws5" Jul 08 04:18:47.972266 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972189 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e2812fd7-bf09-4088-b1eb-67b5ac18f380-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:47.972266 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972212 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e2812fd7-bf09-4088-b1eb-67b5ac18f380-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:47.972266 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972232 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5331e3cb-cabb-4cd9-b7ca-5977c7acaab7-host\") pod \"node-ca-4rws5\" (UID: \"5331e3cb-cabb-4cd9-b7ca-5977c7acaab7\") " pod="openshift-image-registry/node-ca-4rws5" Jul 08 04:18:47.972410 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972293 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-system-cni-dir\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.972410 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972327 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-cnibin\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.972410 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972363 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-var-lib-cni-bin\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.972410 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972393 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5331e3cb-cabb-4cd9-b7ca-5977c7acaab7-serviceca\") pod \"node-ca-4rws5\" (UID: \"5331e3cb-cabb-4cd9-b7ca-5977c7acaab7\") " pod="openshift-image-registry/node-ca-4rws5" Jul 08 04:18:47.972528 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972421 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e2812fd7-bf09-4088-b1eb-67b5ac18f380-os-release\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:47.972528 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972452 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-multus-cni-dir\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.972528 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972475 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-var-lib-cni-multus\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.972528 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972499 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-etc-kubernetes\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.972528 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972519 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e2812fd7-bf09-4088-b1eb-67b5ac18f380-system-cni-dir\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:47.972681 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972534 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e2812fd7-bf09-4088-b1eb-67b5ac18f380-cnibin\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:47.972681 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972549 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e2812fd7-bf09-4088-b1eb-67b5ac18f380-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:47.972681 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972583 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6763059-f532-47e0-8523-50893eb5b3db-cni-binary-copy\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.972681 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972597 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-run-netns\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.972681 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972612 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-multus-conf-dir\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.972681 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972643 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e6763059-f532-47e0-8523-50893eb5b3db-multus-daemon-config\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.972905 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972676 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp756\" (UniqueName: \"kubernetes.io/projected/e6763059-f532-47e0-8523-50893eb5b3db-kube-api-access-hp756\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.972905 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972720 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e2812fd7-bf09-4088-b1eb-67b5ac18f380-cni-binary-copy\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:47.972905 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972751 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6fc503c2-f8af-433e-8450-0f18d2761150-iptables-alerter-script\") pod \"iptables-alerter-58pdq\" (UID: \"6fc503c2-f8af-433e-8450-0f18d2761150\") " pod="openshift-network-operator/iptables-alerter-58pdq" Jul 08 04:18:47.972905 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972778 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fc503c2-f8af-433e-8450-0f18d2761150-host-slash\") pod \"iptables-alerter-58pdq\" (UID: \"6fc503c2-f8af-433e-8450-0f18d2761150\") " pod="openshift-network-operator/iptables-alerter-58pdq" Jul 08 04:18:47.972905 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972796 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp8j2\" (UniqueName: \"kubernetes.io/projected/6fc503c2-f8af-433e-8450-0f18d2761150-kube-api-access-dp8j2\") pod \"iptables-alerter-58pdq\" (UID: \"6fc503c2-f8af-433e-8450-0f18d2761150\") " pod="openshift-network-operator/iptables-alerter-58pdq" Jul 08 04:18:47.972905 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972812 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndgh4\" (UniqueName: \"kubernetes.io/projected/e2812fd7-bf09-4088-b1eb-67b5ac18f380-kube-api-access-ndgh4\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:47.972905 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972839 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-multus-socket-dir-parent\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.972905 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972871 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-hostroot\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.972905 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972887 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-run-multus-certs\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.973209 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972943 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-var-lib-kubelet\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.973209 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972969 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-os-release\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.973209 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.972996 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-run-k8s-cni-cncf-io\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:47.998965 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:47.998945 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-vc52k"] Jul 08 04:18:48.000909 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.000893 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67"] Jul 08 04:18:48.001017 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.001002 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:18:48.001086 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.001059 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:18:48.003056 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.003036 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.005752 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.005732 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cmrjf\"" Jul 08 04:18:48.005838 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.005774 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 04:18:48.005838 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.005785 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 04:18:48.005838 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.005789 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 04:18:48.074127 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074069 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e6763059-f532-47e0-8523-50893eb5b3db-multus-daemon-config\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.074127 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074095 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp756\" (UniqueName: \"kubernetes.io/projected/e6763059-f532-47e0-8523-50893eb5b3db-kube-api-access-hp756\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.074127 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074113 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e2812fd7-bf09-4088-b1eb-67b5ac18f380-cni-binary-copy\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.074327 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074128 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6fc503c2-f8af-433e-8450-0f18d2761150-iptables-alerter-script\") pod \"iptables-alerter-58pdq\" (UID: \"6fc503c2-f8af-433e-8450-0f18d2761150\") " pod="openshift-network-operator/iptables-alerter-58pdq" Jul 08 04:18:48.074327 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074151 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fc503c2-f8af-433e-8450-0f18d2761150-host-slash\") pod \"iptables-alerter-58pdq\" (UID: \"6fc503c2-f8af-433e-8450-0f18d2761150\") " pod="openshift-network-operator/iptables-alerter-58pdq" Jul 08 04:18:48.074327 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074203 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fc503c2-f8af-433e-8450-0f18d2761150-host-slash\") pod \"iptables-alerter-58pdq\" (UID: \"6fc503c2-f8af-433e-8450-0f18d2761150\") " pod="openshift-network-operator/iptables-alerter-58pdq" Jul 08 04:18:48.074327 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074276 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp8j2\" (UniqueName: \"kubernetes.io/projected/6fc503c2-f8af-433e-8450-0f18d2761150-kube-api-access-dp8j2\") pod \"iptables-alerter-58pdq\" (UID: \"6fc503c2-f8af-433e-8450-0f18d2761150\") " pod="openshift-network-operator/iptables-alerter-58pdq" Jul 08 04:18:48.074327 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074316 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndgh4\" (UniqueName: \"kubernetes.io/projected/e2812fd7-bf09-4088-b1eb-67b5ac18f380-kube-api-access-ndgh4\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.074590 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074372 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-multus-socket-dir-parent\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.074590 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074403 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-hostroot\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.074590 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074429 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-run-multus-certs\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.074590 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074452 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-var-lib-kubelet\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.074590 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074479 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03875a1c-3c96-419a-88a5-631347a6cd00-hosts-file\") pod \"node-resolver-slm66\" (UID: \"03875a1c-3c96-419a-88a5-631347a6cd00\") " pod="openshift-dns/node-resolver-slm66" Jul 08 04:18:48.074590 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074511 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-os-release\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.074590 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074519 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-multus-socket-dir-parent\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.074590 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074509 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-hostroot\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.074590 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074536 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-run-k8s-cni-cncf-io\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.074590 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074541 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-run-multus-certs\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.074590 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074552 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-var-lib-kubelet\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.074590 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074579 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-run-k8s-cni-cncf-io\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.074590 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074596 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbn4t\" (UniqueName: \"kubernetes.io/projected/5331e3cb-cabb-4cd9-b7ca-5977c7acaab7-kube-api-access-fbn4t\") pod \"node-ca-4rws5\" (UID: \"5331e3cb-cabb-4cd9-b7ca-5977c7acaab7\") " pod="openshift-image-registry/node-ca-4rws5" Jul 08 04:18:48.075163 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074600 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-os-release\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.075163 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074615 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e2812fd7-bf09-4088-b1eb-67b5ac18f380-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.075163 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074635 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e2812fd7-bf09-4088-b1eb-67b5ac18f380-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.075163 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074654 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5331e3cb-cabb-4cd9-b7ca-5977c7acaab7-host\") pod \"node-ca-4rws5\" (UID: \"5331e3cb-cabb-4cd9-b7ca-5977c7acaab7\") " pod="openshift-image-registry/node-ca-4rws5" Jul 08 04:18:48.075163 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074677 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrffv\" (UniqueName: \"kubernetes.io/projected/03875a1c-3c96-419a-88a5-631347a6cd00-kube-api-access-qrffv\") pod \"node-resolver-slm66\" (UID: \"03875a1c-3c96-419a-88a5-631347a6cd00\") " pod="openshift-dns/node-resolver-slm66" Jul 08 04:18:48.075163 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074702 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-system-cni-dir\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.075163 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074724 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-cnibin\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.075163 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074726 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5331e3cb-cabb-4cd9-b7ca-5977c7acaab7-host\") pod \"node-ca-4rws5\" (UID: \"5331e3cb-cabb-4cd9-b7ca-5977c7acaab7\") " pod="openshift-image-registry/node-ca-4rws5" Jul 08 04:18:48.075163 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074739 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-var-lib-cni-bin\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.075163 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074741 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6fc503c2-f8af-433e-8450-0f18d2761150-iptables-alerter-script\") pod \"iptables-alerter-58pdq\" (UID: \"6fc503c2-f8af-433e-8450-0f18d2761150\") " pod="openshift-network-operator/iptables-alerter-58pdq" Jul 08 04:18:48.075163 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074753 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5331e3cb-cabb-4cd9-b7ca-5977c7acaab7-serviceca\") pod \"node-ca-4rws5\" (UID: \"5331e3cb-cabb-4cd9-b7ca-5977c7acaab7\") " pod="openshift-image-registry/node-ca-4rws5" Jul 08 04:18:48.075163 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074773 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-system-cni-dir\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.075163 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074790 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-var-lib-cni-bin\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.075163 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074805 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-cnibin\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.075163 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074804 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03875a1c-3c96-419a-88a5-631347a6cd00-tmp-dir\") pod \"node-resolver-slm66\" (UID: \"03875a1c-3c96-419a-88a5-631347a6cd00\") " pod="openshift-dns/node-resolver-slm66" Jul 08 04:18:48.075714 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074835 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/986153ab-266b-49f8-846e-cdde4d6b90af-agent-certs\") pod \"konnectivity-agent-m5qfn\" (UID: \"986153ab-266b-49f8-846e-cdde4d6b90af\") " pod="kube-system/konnectivity-agent-m5qfn" Jul 08 04:18:48.075714 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074854 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/986153ab-266b-49f8-846e-cdde4d6b90af-konnectivity-ca\") pod \"konnectivity-agent-m5qfn\" (UID: \"986153ab-266b-49f8-846e-cdde4d6b90af\") " pod="kube-system/konnectivity-agent-m5qfn" Jul 08 04:18:48.075714 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074882 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e2812fd7-bf09-4088-b1eb-67b5ac18f380-os-release\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.075714 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074903 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-multus-cni-dir\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.075714 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074919 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-var-lib-cni-multus\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.075714 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074933 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-etc-kubernetes\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.075714 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074951 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e2812fd7-bf09-4088-b1eb-67b5ac18f380-system-cni-dir\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.075714 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074973 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e2812fd7-bf09-4088-b1eb-67b5ac18f380-os-release\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.075714 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074976 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-multus-cni-dir\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.075714 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074974 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e2812fd7-bf09-4088-b1eb-67b5ac18f380-cnibin\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.075714 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.074999 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-var-lib-cni-multus\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.075714 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.075009 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e2812fd7-bf09-4088-b1eb-67b5ac18f380-cnibin\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.075714 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.075024 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e2812fd7-bf09-4088-b1eb-67b5ac18f380-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.075714 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.075041 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-etc-kubernetes\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.076083 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.075053 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6763059-f532-47e0-8523-50893eb5b3db-cni-binary-copy\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.076083 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.075072 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-run-netns\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.076083 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.075053 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e2812fd7-bf09-4088-b1eb-67b5ac18f380-system-cni-dir\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.076083 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.075089 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-multus-conf-dir\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.076083 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.075125 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-multus-conf-dir\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.076083 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.075132 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e6763059-f532-47e0-8523-50893eb5b3db-host-run-netns\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.076083 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.075261 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e2812fd7-bf09-4088-b1eb-67b5ac18f380-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.076083 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.075413 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e2812fd7-bf09-4088-b1eb-67b5ac18f380-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.076083 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.075434 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e2812fd7-bf09-4088-b1eb-67b5ac18f380-cni-binary-copy\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.076083 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.075450 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5331e3cb-cabb-4cd9-b7ca-5977c7acaab7-serviceca\") pod \"node-ca-4rws5\" (UID: \"5331e3cb-cabb-4cd9-b7ca-5977c7acaab7\") " pod="openshift-image-registry/node-ca-4rws5" Jul 08 04:18:48.076083 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.075516 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e2812fd7-bf09-4088-b1eb-67b5ac18f380-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.076083 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.075598 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e6763059-f532-47e0-8523-50893eb5b3db-multus-daemon-config\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.076083 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.075600 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e6763059-f532-47e0-8523-50893eb5b3db-cni-binary-copy\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.080949 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.080930 2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 04:18:48.084537 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.084510 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndgh4\" (UniqueName: \"kubernetes.io/projected/e2812fd7-bf09-4088-b1eb-67b5ac18f380-kube-api-access-ndgh4\") pod \"multus-additional-cni-plugins-s4rvj\" (UID: \"e2812fd7-bf09-4088-b1eb-67b5ac18f380\") " pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.084537 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.084521 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp8j2\" (UniqueName: \"kubernetes.io/projected/6fc503c2-f8af-433e-8450-0f18d2761150-kube-api-access-dp8j2\") pod \"iptables-alerter-58pdq\" (UID: \"6fc503c2-f8af-433e-8450-0f18d2761150\") " pod="openshift-network-operator/iptables-alerter-58pdq" Jul 08 04:18:48.084655 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.084525 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbn4t\" (UniqueName: \"kubernetes.io/projected/5331e3cb-cabb-4cd9-b7ca-5977c7acaab7-kube-api-access-fbn4t\") pod \"node-ca-4rws5\" (UID: \"5331e3cb-cabb-4cd9-b7ca-5977c7acaab7\") " pod="openshift-image-registry/node-ca-4rws5" Jul 08 04:18:48.084655 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.084613 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp756\" (UniqueName: \"kubernetes.io/projected/e6763059-f532-47e0-8523-50893eb5b3db-kube-api-access-hp756\") pod \"multus-f545t\" (UID: \"e6763059-f532-47e0-8523-50893eb5b3db\") " pod="openshift-multus/multus-f545t" Jul 08 04:18:48.095208 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.095192 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-qthv2"] Jul 08 04:18:48.098656 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.098638 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-zc555"] Jul 08 04:18:48.098752 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.098738 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.100822 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.100806 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.101301 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.101286 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 04:18:48.101301 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.101296 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-s2r6g\"" Jul 08 04:18:48.101448 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.101296 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 04:18:48.101448 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.101285 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 04:18:48.101448 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.101324 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 04:18:48.102611 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.102596 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 04:18:48.103007 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.102993 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 04:18:48.104225 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.104202 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 04:18:48.104336 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.104213 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 04:18:48.104336 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.104233 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-twj9h\"" Jul 08 04:18:48.111181 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.111165 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:18:48.111565 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.111545 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" event={"ID":"26ba4c92c69564f7ea9cda441d8bf34b","Type":"ContainerStarted","Data":"6c1afdf606837c3d2f04c62d285d56a2eef1643f4fe07fb039d041b57d9932ee"} Jul 08 04:18:48.113899 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.113884 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f545t" Jul 08 04:18:48.117200 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.117186 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-6j45q"] Jul 08 04:18:48.119589 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.119569 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-58pdq" Jul 08 04:18:48.119681 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.119665 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-58wwx"] Jul 08 04:18:48.119781 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.119767 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:18:48.119850 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.119833 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:18:48.122308 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:48.122278 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6763059_f532_47e0_8523_50893eb5b3db.slice/crio-019add0da60b42b48249140f16168510a3cd2fedfd8b0dcd1da2048990f126f1 WatchSource:0}: Error finding container 019add0da60b42b48249140f16168510a3cd2fedfd8b0dcd1da2048990f126f1: Status 404 returned error can't find the container with id 019add0da60b42b48249140f16168510a3cd2fedfd8b0dcd1da2048990f126f1 Jul 08 04:18:48.122378 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.122367 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:48.122452 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.122429 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:18:48.128178 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:48.128157 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fc503c2_f8af_433e_8450_0f18d2761150.slice/crio-0f63159f1c8bed6dae3f1e103a3e38db777f0f47e51e9ac4b065d91ab9353971 WatchSource:0}: Error finding container 0f63159f1c8bed6dae3f1e103a3e38db777f0f47e51e9ac4b065d91ab9353971: Status 404 returned error can't find the container with id 0f63159f1c8bed6dae3f1e103a3e38db777f0f47e51e9ac4b065d91ab9353971 Jul 08 04:18:48.132324 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.132309 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4rws5" Jul 08 04:18:48.135483 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.135466 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s4rvj" Jul 08 04:18:48.142061 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:48.142033 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5331e3cb_cabb_4cd9_b7ca_5977c7acaab7.slice/crio-2d9c06fed22b191c05e5a03065d320ed7b38e19c257d12edf02a1dc0ccc7303a WatchSource:0}: Error finding container 2d9c06fed22b191c05e5a03065d320ed7b38e19c257d12edf02a1dc0ccc7303a: Status 404 returned error can't find the container with id 2d9c06fed22b191c05e5a03065d320ed7b38e19c257d12edf02a1dc0ccc7303a Jul 08 04:18:48.145028 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:48.145010 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2812fd7_bf09_4088_b1eb_67b5ac18f380.slice/crio-1d5135d3e298d57e1c9fcee4935ca970f4685c57d1d4edd059ffb7d46be9fb39 WatchSource:0}: Error finding container 1d5135d3e298d57e1c9fcee4935ca970f4685c57d1d4edd059ffb7d46be9fb39: Status 404 returned error can't find the container with id 1d5135d3e298d57e1c9fcee4935ca970f4685c57d1d4edd059ffb7d46be9fb39 Jul 08 04:18:48.157673 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.157639 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal" podStartSLOduration=1.157628284 podStartE2EDuration="1.157628284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:18:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:48.157242224 +0000 UTC m=+20.594017204" watchObservedRunningTime="2026-07-08 04:18:48.157628284 +0000 UTC m=+20.594403264" Jul 08 04:18:48.175549 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.175532 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-sys-fs\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.175617 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.175556 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpt6j\" (UniqueName: \"kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j\") pod \"network-check-target-vc52k\" (UID: \"bee553af-478a-4f02-8236-787b25adaadc\") " pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:18:48.175617 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.175576 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrffv\" (UniqueName: \"kubernetes.io/projected/03875a1c-3c96-419a-88a5-631347a6cd00-kube-api-access-qrffv\") pod \"node-resolver-slm66\" (UID: \"03875a1c-3c96-419a-88a5-631347a6cd00\") " pod="openshift-dns/node-resolver-slm66" Jul 08 04:18:48.175617 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.175591 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-etc-selinux\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.175617 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.175608 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03875a1c-3c96-419a-88a5-631347a6cd00-tmp-dir\") pod \"node-resolver-slm66\" (UID: \"03875a1c-3c96-419a-88a5-631347a6cd00\") " pod="openshift-dns/node-resolver-slm66" Jul 08 04:18:48.175806 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.175626 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/986153ab-266b-49f8-846e-cdde4d6b90af-agent-certs\") pod \"konnectivity-agent-m5qfn\" (UID: \"986153ab-266b-49f8-846e-cdde4d6b90af\") " pod="kube-system/konnectivity-agent-m5qfn" Jul 08 04:18:48.175806 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.175647 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/986153ab-266b-49f8-846e-cdde4d6b90af-konnectivity-ca\") pod \"konnectivity-agent-m5qfn\" (UID: \"986153ab-266b-49f8-846e-cdde4d6b90af\") " pod="kube-system/konnectivity-agent-m5qfn" Jul 08 04:18:48.175889 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.175853 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.175889 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.175881 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-socket-dir\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.175974 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.175905 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-device-dir\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.175974 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.175933 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l5lb\" (UniqueName: \"kubernetes.io/projected/7b3c4c01-4894-4267-8e0d-418451b980ff-kube-api-access-8l5lb\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.175974 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.175961 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03875a1c-3c96-419a-88a5-631347a6cd00-hosts-file\") pod \"node-resolver-slm66\" (UID: \"03875a1c-3c96-419a-88a5-631347a6cd00\") " pod="openshift-dns/node-resolver-slm66" Jul 08 04:18:48.176069 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.175986 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-registration-dir\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.176069 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.176016 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03875a1c-3c96-419a-88a5-631347a6cd00-hosts-file\") pod \"node-resolver-slm66\" (UID: \"03875a1c-3c96-419a-88a5-631347a6cd00\") " pod="openshift-dns/node-resolver-slm66" Jul 08 04:18:48.176129 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.176090 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03875a1c-3c96-419a-88a5-631347a6cd00-tmp-dir\") pod \"node-resolver-slm66\" (UID: \"03875a1c-3c96-419a-88a5-631347a6cd00\") " pod="openshift-dns/node-resolver-slm66" Jul 08 04:18:48.176855 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.176840 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/986153ab-266b-49f8-846e-cdde4d6b90af-konnectivity-ca\") pod \"konnectivity-agent-m5qfn\" (UID: \"986153ab-266b-49f8-846e-cdde4d6b90af\") " pod="kube-system/konnectivity-agent-m5qfn" Jul 08 04:18:48.177836 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.177820 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/986153ab-266b-49f8-846e-cdde4d6b90af-agent-certs\") pod \"konnectivity-agent-m5qfn\" (UID: \"986153ab-266b-49f8-846e-cdde4d6b90af\") " pod="kube-system/konnectivity-agent-m5qfn" Jul 08 04:18:48.182717 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.182702 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrffv\" (UniqueName: \"kubernetes.io/projected/03875a1c-3c96-419a-88a5-631347a6cd00-kube-api-access-qrffv\") pod \"node-resolver-slm66\" (UID: \"03875a1c-3c96-419a-88a5-631347a6cd00\") " pod="openshift-dns/node-resolver-slm66" Jul 08 04:18:48.215598 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.215568 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-slm66" Jul 08 04:18:48.225922 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:48.225903 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03875a1c_3c96_419a_88a5_631347a6cd00.slice/crio-ab33891375650c64f0b22384b1af7ee2e86b39e6a3c3fb89a3e67edc911ca7d9 WatchSource:0}: Error finding container ab33891375650c64f0b22384b1af7ee2e86b39e6a3c3fb89a3e67edc911ca7d9: Status 404 returned error can't find the container with id ab33891375650c64f0b22384b1af7ee2e86b39e6a3c3fb89a3e67edc911ca7d9 Jul 08 04:18:48.226712 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.226699 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m5qfn" Jul 08 04:18:48.234302 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:48.234283 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod986153ab_266b_49f8_846e_cdde4d6b90af.slice/crio-ec9add0e11615ce29728263721b123944fe518495d106ec6c2487a23446c6eee WatchSource:0}: Error finding container ec9add0e11615ce29728263721b123944fe518495d106ec6c2487a23446c6eee: Status 404 returned error can't find the container with id ec9add0e11615ce29728263721b123944fe518495d106ec6c2487a23446c6eee Jul 08 04:18:48.276853 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.276823 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-host\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.276933 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.276855 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-log-socket\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.276933 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.276873 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-cni-netd\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.276933 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.276889 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c45nt\" (UniqueName: \"kubernetes.io/projected/37c1fce9-b855-4864-a4e0-04b78dd27b62-kube-api-access-c45nt\") pod \"network-metrics-daemon-6j45q\" (UID: \"37c1fce9-b855-4864-a4e0-04b78dd27b62\") " pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:18:48.277017 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.276938 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l5lb\" (UniqueName: \"kubernetes.io/projected/7b3c4c01-4894-4267-8e0d-418451b980ff-kube-api-access-8l5lb\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.277017 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.276968 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9afcd46e-0ea4-4870-98ca-6f18f177f324-ovnkube-config\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.277084 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277063 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9afcd46e-0ea4-4870-98ca-6f18f177f324-ovn-node-metrics-cert\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.277112 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277090 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-modprobe-d\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.277140 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277111 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-registration-dir\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.277140 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277127 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-sys-fs\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.277196 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277157 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.277196 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277179 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9afcd46e-0ea4-4870-98ca-6f18f177f324-env-overrides\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.277196 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277181 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-registration-dir\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.277277 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277208 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-var-lib-openvswitch\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.277277 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277225 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-sys-fs\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.277277 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277232 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs\") pod \"network-metrics-daemon-6j45q\" (UID: \"37c1fce9-b855-4864-a4e0-04b78dd27b62\") " pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:18:48.277277 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277261 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-sysctl-conf\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.277411 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277292 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-lib-modules\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.277411 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277309 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b878e496-2a4c-41db-9efd-756f89197d14-tmp\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.277411 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277323 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-run-netns\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.277411 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277336 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b878e496-2a4c-41db-9efd-756f89197d14-etc-tuned\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.277411 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277368 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/471fcdae-c4d6-4999-b260-0f0719e99682-dbus\") pod \"global-pull-secret-syncer-58wwx\" (UID: \"471fcdae-c4d6-4999-b260-0f0719e99682\") " pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:48.277411 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277385 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-sys\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.277411 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277405 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.277595 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277419 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-run-openvswitch\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.277595 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277434 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-socket-dir\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.277595 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277448 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-systemd-units\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.277595 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277471 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-run-systemd\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.277595 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277493 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.277595 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277508 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-cni-bin\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.277595 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277536 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-var-lib-kubelet\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.277595 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277551 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-etc-openvswitch\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.277595 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277568 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-socket-dir\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.277595 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277572 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-device-dir\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.277857 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277610 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-device-dir\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.277857 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277632 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-sysconfig\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.277857 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277701 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-run-ovn\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.277857 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277728 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-systemd\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.277857 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277746 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxslj\" (UniqueName: \"kubernetes.io/projected/b878e496-2a4c-41db-9efd-756f89197d14-kube-api-access-kxslj\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.277857 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277778 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpt6j\" (UniqueName: \"kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j\") pod \"network-check-target-vc52k\" (UID: \"bee553af-478a-4f02-8236-787b25adaadc\") " pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:18:48.277857 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277796 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-etc-selinux\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.277857 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277817 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-node-log\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.277857 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277840 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret\") pod \"global-pull-secret-syncer-58wwx\" (UID: \"471fcdae-c4d6-4999-b260-0f0719e99682\") " pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:48.278118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277882 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-run-ovn-kubernetes\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.278118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277905 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9afcd46e-0ea4-4870-98ca-6f18f177f324-ovnkube-script-lib\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.278118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277909 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b3c4c01-4894-4267-8e0d-418451b980ff-etc-selinux\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.278118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277928 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-kubernetes\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.278118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277951 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-slash\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.278118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277973 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6jck\" (UniqueName: \"kubernetes.io/projected/9afcd46e-0ea4-4870-98ca-6f18f177f324-kube-api-access-z6jck\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.278118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.277993 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-sysctl-d\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.278118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.278018 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/471fcdae-c4d6-4999-b260-0f0719e99682-kubelet-config\") pod \"global-pull-secret-syncer-58wwx\" (UID: \"471fcdae-c4d6-4999-b260-0f0719e99682\") " pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:48.278118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.278038 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-kubelet\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.278118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.278050 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-run\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.283339 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.283324 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 04:18:48.283400 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.283357 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 04:18:48.283400 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.283366 2583 projected.go:196] Error preparing data for projected volume kube-api-access-lpt6j for pod openshift-network-diagnostics/network-check-target-vc52k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:18:48.283460 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.283418 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j podName:bee553af-478a-4f02-8236-787b25adaadc nodeName:}" failed. No retries permitted until 2026-07-08 04:18:48.783399916 +0000 UTC m=+21.220174877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lpt6j" (UniqueName: "kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j") pod "network-check-target-vc52k" (UID: "bee553af-478a-4f02-8236-787b25adaadc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:18:48.286017 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.286002 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l5lb\" (UniqueName: \"kubernetes.io/projected/7b3c4c01-4894-4267-8e0d-418451b980ff-kube-api-access-8l5lb\") pod \"aws-ebs-csi-driver-node-45n67\" (UID: \"7b3c4c01-4894-4267-8e0d-418451b980ff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.310926 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.310904 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" Jul 08 04:18:48.320270 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:48.320238 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b3c4c01_4894_4267_8e0d_418451b980ff.slice/crio-9978ba6d7940e61cf7cf71b89d39a2a06996d938df0d7f51bd422836336dc5a1 WatchSource:0}: Error finding container 9978ba6d7940e61cf7cf71b89d39a2a06996d938df0d7f51bd422836336dc5a1: Status 404 returned error can't find the container with id 9978ba6d7940e61cf7cf71b89d39a2a06996d938df0d7f51bd422836336dc5a1 Jul 08 04:18:48.378559 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378477 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-sys\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.378559 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378534 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-run-openvswitch\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.378559 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378552 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-systemd-units\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.378748 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378591 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-systemd-units\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.378748 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378598 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-run-openvswitch\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.378748 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378661 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-run-systemd\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.378748 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378684 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-cni-bin\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.378748 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378684 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-sys\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.378748 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378701 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-var-lib-kubelet\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.378748 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378743 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-run-systemd\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.379038 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378742 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-cni-bin\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.379038 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378760 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-var-lib-kubelet\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.379038 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378785 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-etc-openvswitch\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.379038 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378805 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-sysconfig\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.379038 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378819 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-run-ovn\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.379038 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378835 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-systemd\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.379038 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378851 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxslj\" (UniqueName: \"kubernetes.io/projected/b878e496-2a4c-41db-9efd-756f89197d14-kube-api-access-kxslj\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.379038 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378871 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-run-ovn\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.379038 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378888 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-etc-openvswitch\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.379038 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378913 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-sysconfig\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.379038 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378929 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-node-log\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.379038 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378929 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-systemd\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.379038 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378952 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret\") pod \"global-pull-secret-syncer-58wwx\" (UID: \"471fcdae-c4d6-4999-b260-0f0719e99682\") " pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:48.379038 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378972 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-node-log\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.379038 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.378978 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-run-ovn-kubernetes\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.379743 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379002 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9afcd46e-0ea4-4870-98ca-6f18f177f324-ovnkube-script-lib\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.379743 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379023 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-kubernetes\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.379743 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379046 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-slash\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.379743 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.379057 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 04:18:48.379743 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379060 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-run-ovn-kubernetes\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.379743 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379107 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6jck\" (UniqueName: \"kubernetes.io/projected/9afcd46e-0ea4-4870-98ca-6f18f177f324-kube-api-access-z6jck\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.379743 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379123 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-kubernetes\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.379743 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379127 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-sysctl-d\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.379743 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379144 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-slash\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.379743 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.379163 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret podName:471fcdae-c4d6-4999-b260-0f0719e99682 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:48.879140113 +0000 UTC m=+21.315915085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret") pod "global-pull-secret-syncer-58wwx" (UID: "471fcdae-c4d6-4999-b260-0f0719e99682") : object "kube-system"/"original-pull-secret" not registered Jul 08 04:18:48.379743 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379220 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/471fcdae-c4d6-4999-b260-0f0719e99682-kubelet-config\") pod \"global-pull-secret-syncer-58wwx\" (UID: \"471fcdae-c4d6-4999-b260-0f0719e99682\") " pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:48.379743 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379243 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-kubelet\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.379743 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379264 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-run\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.379743 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379284 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/471fcdae-c4d6-4999-b260-0f0719e99682-kubelet-config\") pod \"global-pull-secret-syncer-58wwx\" (UID: \"471fcdae-c4d6-4999-b260-0f0719e99682\") " pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:48.380403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379292 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-host\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.380403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379300 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-kubelet\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.380403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379326 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-log-socket\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.380403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379357 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-host\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.380403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379368 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-sysctl-d\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.380403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379383 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-log-socket\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.380403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379371 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-cni-netd\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.380403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379376 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-run\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.380403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379400 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-cni-netd\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.380403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379418 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45nt\" (UniqueName: \"kubernetes.io/projected/37c1fce9-b855-4864-a4e0-04b78dd27b62-kube-api-access-c45nt\") pod \"network-metrics-daemon-6j45q\" (UID: \"37c1fce9-b855-4864-a4e0-04b78dd27b62\") " pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:18:48.380403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379449 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9afcd46e-0ea4-4870-98ca-6f18f177f324-ovnkube-config\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.380403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379471 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9afcd46e-0ea4-4870-98ca-6f18f177f324-ovn-node-metrics-cert\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.380403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379494 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-modprobe-d\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.380403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379524 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.380403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379539 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9afcd46e-0ea4-4870-98ca-6f18f177f324-ovnkube-script-lib\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.381142 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379551 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9afcd46e-0ea4-4870-98ca-6f18f177f324-env-overrides\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.381142 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379585 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-var-lib-openvswitch\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.381142 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379584 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.381142 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379622 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-modprobe-d\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.381142 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379630 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-var-lib-openvswitch\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.381142 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379645 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs\") pod \"network-metrics-daemon-6j45q\" (UID: \"37c1fce9-b855-4864-a4e0-04b78dd27b62\") " pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:18:48.381142 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379679 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-sysctl-conf\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.381142 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379702 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-lib-modules\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.381142 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.379715 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:18:48.381142 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.379756 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs podName:37c1fce9-b855-4864-a4e0-04b78dd27b62 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:48.879741416 +0000 UTC m=+21.316516388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs") pod "network-metrics-daemon-6j45q" (UID: "37c1fce9-b855-4864-a4e0-04b78dd27b62") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:18:48.381142 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379781 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b878e496-2a4c-41db-9efd-756f89197d14-tmp\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.381142 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379806 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-lib-modules\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.381142 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379808 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-run-netns\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.381142 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379859 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b878e496-2a4c-41db-9efd-756f89197d14-etc-tuned\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.381811 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379867 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9afcd46e-0ea4-4870-98ca-6f18f177f324-host-run-netns\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.381811 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379879 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b878e496-2a4c-41db-9efd-756f89197d14-etc-sysctl-conf\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.381811 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379885 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/471fcdae-c4d6-4999-b260-0f0719e99682-dbus\") pod \"global-pull-secret-syncer-58wwx\" (UID: \"471fcdae-c4d6-4999-b260-0f0719e99682\") " pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:48.381811 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379949 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9afcd46e-0ea4-4870-98ca-6f18f177f324-ovnkube-config\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.381811 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.379985 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/471fcdae-c4d6-4999-b260-0f0719e99682-dbus\") pod \"global-pull-secret-syncer-58wwx\" (UID: \"471fcdae-c4d6-4999-b260-0f0719e99682\") " pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:48.381811 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.380423 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9afcd46e-0ea4-4870-98ca-6f18f177f324-env-overrides\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.381992 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.381832 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b878e496-2a4c-41db-9efd-756f89197d14-tmp\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.381992 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.381870 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b878e496-2a4c-41db-9efd-756f89197d14-etc-tuned\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.382175 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.382156 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9afcd46e-0ea4-4870-98ca-6f18f177f324-ovn-node-metrics-cert\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.389646 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.389628 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxslj\" (UniqueName: \"kubernetes.io/projected/b878e496-2a4c-41db-9efd-756f89197d14-kube-api-access-kxslj\") pod \"tuned-zc555\" (UID: \"b878e496-2a4c-41db-9efd-756f89197d14\") " pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.389742 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.389674 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6jck\" (UniqueName: \"kubernetes.io/projected/9afcd46e-0ea4-4870-98ca-6f18f177f324-kube-api-access-z6jck\") pod \"ovnkube-node-qthv2\" (UID: \"9afcd46e-0ea4-4870-98ca-6f18f177f324\") " pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.389878 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.389862 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c45nt\" (UniqueName: \"kubernetes.io/projected/37c1fce9-b855-4864-a4e0-04b78dd27b62-kube-api-access-c45nt\") pod \"network-metrics-daemon-6j45q\" (UID: \"37c1fce9-b855-4864-a4e0-04b78dd27b62\") " pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:18:48.407410 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.407394 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:18:48.412243 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.412223 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zc555" Jul 08 04:18:48.416924 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:48.416905 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9afcd46e_0ea4_4870_98ca_6f18f177f324.slice/crio-5c5825559dfcb83a5f6bbdc7d9215a142cb79d3268c3ac7232d9a397228624cf WatchSource:0}: Error finding container 5c5825559dfcb83a5f6bbdc7d9215a142cb79d3268c3ac7232d9a397228624cf: Status 404 returned error can't find the container with id 5c5825559dfcb83a5f6bbdc7d9215a142cb79d3268c3ac7232d9a397228624cf Jul 08 04:18:48.420574 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:18:48.420555 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb878e496_2a4c_41db_9efd_756f89197d14.slice/crio-0868daced621e8a250440ee2796827b5a45abeddf36b1bfd7b4f5e9d9bd813cc WatchSource:0}: Error finding container 0868daced621e8a250440ee2796827b5a45abeddf36b1bfd7b4f5e9d9bd813cc: Status 404 returned error can't find the container with id 0868daced621e8a250440ee2796827b5a45abeddf36b1bfd7b4f5e9d9bd813cc Jul 08 04:18:48.784291 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.784253 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpt6j\" (UniqueName: \"kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j\") pod \"network-check-target-vc52k\" (UID: \"bee553af-478a-4f02-8236-787b25adaadc\") " pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:18:48.784490 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.784438 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 04:18:48.784490 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.784463 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 04:18:48.784490 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.784477 2583 projected.go:196] Error preparing data for projected volume kube-api-access-lpt6j for pod openshift-network-diagnostics/network-check-target-vc52k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:18:48.784657 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.784536 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j podName:bee553af-478a-4f02-8236-787b25adaadc nodeName:}" failed. No retries permitted until 2026-07-08 04:18:49.784517364 +0000 UTC m=+22.221292334 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpt6j" (UniqueName: "kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j") pod "network-check-target-vc52k" (UID: "bee553af-478a-4f02-8236-787b25adaadc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:18:48.884992 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.884958 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret\") pod \"global-pull-secret-syncer-58wwx\" (UID: \"471fcdae-c4d6-4999-b260-0f0719e99682\") " pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:48.885147 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.885028 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs\") pod \"network-metrics-daemon-6j45q\" (UID: \"37c1fce9-b855-4864-a4e0-04b78dd27b62\") " pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:18:48.885219 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.885153 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:18:48.885219 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.885212 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs podName:37c1fce9-b855-4864-a4e0-04b78dd27b62 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:49.885194267 +0000 UTC m=+22.321969233 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs") pod "network-metrics-daemon-6j45q" (UID: "37c1fce9-b855-4864-a4e0-04b78dd27b62") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:18:48.885617 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.885597 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 04:18:48.885697 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:48.885654 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret podName:471fcdae-c4d6-4999-b260-0f0719e99682 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:49.885637066 +0000 UTC m=+22.322412027 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret") pod "global-pull-secret-syncer-58wwx" (UID: "471fcdae-c4d6-4999-b260-0f0719e99682") : object "kube-system"/"original-pull-secret" not registered Jul 08 04:18:48.935737 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.935677 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 04:13:47 +0000 UTC" deadline="2028-02-29 20:05:01.533591578 +0000 UTC" Jul 08 04:18:48.935737 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:48.935735 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14439h46m12.597859811s" Jul 08 04:18:49.130292 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:49.129662 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" event={"ID":"9afcd46e-0ea4-4870-98ca-6f18f177f324","Type":"ContainerStarted","Data":"5c5825559dfcb83a5f6bbdc7d9215a142cb79d3268c3ac7232d9a397228624cf"} Jul 08 04:18:49.146106 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:49.146066 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" event={"ID":"7b3c4c01-4894-4267-8e0d-418451b980ff","Type":"ContainerStarted","Data":"9978ba6d7940e61cf7cf71b89d39a2a06996d938df0d7f51bd422836336dc5a1"} Jul 08 04:18:49.150492 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:49.150463 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-slm66" event={"ID":"03875a1c-3c96-419a-88a5-631347a6cd00","Type":"ContainerStarted","Data":"ab33891375650c64f0b22384b1af7ee2e86b39e6a3c3fb89a3e67edc911ca7d9"} Jul 08 04:18:49.158569 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:49.158544 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f545t" event={"ID":"e6763059-f532-47e0-8523-50893eb5b3db","Type":"ContainerStarted","Data":"019add0da60b42b48249140f16168510a3cd2fedfd8b0dcd1da2048990f126f1"} Jul 08 04:18:49.172046 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:49.168761 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zc555" event={"ID":"b878e496-2a4c-41db-9efd-756f89197d14","Type":"ContainerStarted","Data":"0868daced621e8a250440ee2796827b5a45abeddf36b1bfd7b4f5e9d9bd813cc"} Jul 08 04:18:49.175894 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:49.175858 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m5qfn" event={"ID":"986153ab-266b-49f8-846e-cdde4d6b90af","Type":"ContainerStarted","Data":"ec9add0e11615ce29728263721b123944fe518495d106ec6c2487a23446c6eee"} Jul 08 04:18:49.186352 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:49.186294 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rvj" event={"ID":"e2812fd7-bf09-4088-b1eb-67b5ac18f380","Type":"ContainerStarted","Data":"1d5135d3e298d57e1c9fcee4935ca970f4685c57d1d4edd059ffb7d46be9fb39"} Jul 08 04:18:49.195950 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:49.195925 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4rws5" event={"ID":"5331e3cb-cabb-4cd9-b7ca-5977c7acaab7","Type":"ContainerStarted","Data":"2d9c06fed22b191c05e5a03065d320ed7b38e19c257d12edf02a1dc0ccc7303a"} Jul 08 04:18:49.204549 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:49.204526 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-58pdq" event={"ID":"6fc503c2-f8af-433e-8450-0f18d2761150","Type":"ContainerStarted","Data":"0f63159f1c8bed6dae3f1e103a3e38db777f0f47e51e9ac4b065d91ab9353971"} Jul 08 04:18:49.791892 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:49.791855 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpt6j\" (UniqueName: \"kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j\") pod \"network-check-target-vc52k\" (UID: \"bee553af-478a-4f02-8236-787b25adaadc\") " pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:18:49.792393 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:49.792042 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 04:18:49.792393 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:49.792063 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 04:18:49.792393 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:49.792076 2583 projected.go:196] Error preparing data for projected volume kube-api-access-lpt6j for pod openshift-network-diagnostics/network-check-target-vc52k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:18:49.792393 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:49.792136 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j podName:bee553af-478a-4f02-8236-787b25adaadc nodeName:}" failed. No retries permitted until 2026-07-08 04:18:51.792118073 +0000 UTC m=+24.228893047 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpt6j" (UniqueName: "kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j") pod "network-check-target-vc52k" (UID: "bee553af-478a-4f02-8236-787b25adaadc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:18:49.892580 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:49.892360 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret\") pod \"global-pull-secret-syncer-58wwx\" (UID: \"471fcdae-c4d6-4999-b260-0f0719e99682\") " pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:49.892580 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:49.892428 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs\") pod \"network-metrics-daemon-6j45q\" (UID: \"37c1fce9-b855-4864-a4e0-04b78dd27b62\") " pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:18:49.892580 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:49.892487 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 04:18:49.892580 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:49.892504 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:18:49.892580 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:49.892547 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs podName:37c1fce9-b855-4864-a4e0-04b78dd27b62 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:51.892529686 +0000 UTC m=+24.329304661 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs") pod "network-metrics-daemon-6j45q" (UID: "37c1fce9-b855-4864-a4e0-04b78dd27b62") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:18:49.892580 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:49.892562 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret podName:471fcdae-c4d6-4999-b260-0f0719e99682 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:51.89255398 +0000 UTC m=+24.329328938 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret") pod "global-pull-secret-syncer-58wwx" (UID: "471fcdae-c4d6-4999-b260-0f0719e99682") : object "kube-system"/"original-pull-secret" not registered Jul 08 04:18:49.936696 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:49.936598 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 04:13:47 +0000 UTC" deadline="2028-04-22 22:36:00.576385348 +0000 UTC" Jul 08 04:18:49.936696 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:49.936640 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15714h17m10.63975087s" Jul 08 04:18:50.071379 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:50.070416 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:18:50.071379 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:50.070542 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:18:50.071379 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:50.070931 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:18:50.071379 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:50.071036 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:18:50.071379 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:50.071094 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:50.071379 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:50.071163 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:18:51.807448 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:51.807316 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpt6j\" (UniqueName: \"kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j\") pod \"network-check-target-vc52k\" (UID: \"bee553af-478a-4f02-8236-787b25adaadc\") " pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:18:51.808067 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:51.807505 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 04:18:51.808067 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:51.807532 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 04:18:51.808067 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:51.807544 2583 projected.go:196] Error preparing data for projected volume kube-api-access-lpt6j for pod openshift-network-diagnostics/network-check-target-vc52k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:18:51.808067 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:51.807608 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j podName:bee553af-478a-4f02-8236-787b25adaadc nodeName:}" failed. No retries permitted until 2026-07-08 04:18:55.807589699 +0000 UTC m=+28.244364658 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpt6j" (UniqueName: "kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j") pod "network-check-target-vc52k" (UID: "bee553af-478a-4f02-8236-787b25adaadc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:18:51.908320 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:51.908120 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret\") pod \"global-pull-secret-syncer-58wwx\" (UID: \"471fcdae-c4d6-4999-b260-0f0719e99682\") " pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:51.908320 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:51.908195 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs\") pod \"network-metrics-daemon-6j45q\" (UID: \"37c1fce9-b855-4864-a4e0-04b78dd27b62\") " pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:18:51.908562 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:51.908339 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 04:18:51.908562 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:51.908427 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret podName:471fcdae-c4d6-4999-b260-0f0719e99682 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:55.908406321 +0000 UTC m=+28.345181295 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret") pod "global-pull-secret-syncer-58wwx" (UID: "471fcdae-c4d6-4999-b260-0f0719e99682") : object "kube-system"/"original-pull-secret" not registered Jul 08 04:18:51.908833 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:51.908735 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:18:51.908833 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:51.908800 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs podName:37c1fce9-b855-4864-a4e0-04b78dd27b62 nodeName:}" failed. No retries permitted until 2026-07-08 04:18:55.908784757 +0000 UTC m=+28.345559716 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs") pod "network-metrics-daemon-6j45q" (UID: "37c1fce9-b855-4864-a4e0-04b78dd27b62") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:18:52.068253 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:52.068170 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:18:52.068427 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:52.068311 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:18:52.068489 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:52.068420 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:18:52.068773 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:52.068544 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:18:52.068773 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:52.068657 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:52.068773 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:52.068737 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:18:54.068775 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:54.068618 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:54.068775 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:54.068615 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:18:54.068775 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:54.068761 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:18:54.069373 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:54.068615 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:18:54.069373 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:54.068834 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:18:54.069373 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:54.068904 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:18:55.839082 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:55.838989 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpt6j\" (UniqueName: \"kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j\") pod \"network-check-target-vc52k\" (UID: \"bee553af-478a-4f02-8236-787b25adaadc\") " pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:18:55.839556 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:55.839191 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 04:18:55.839556 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:55.839214 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 04:18:55.839556 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:55.839226 2583 projected.go:196] Error preparing data for projected volume kube-api-access-lpt6j for pod openshift-network-diagnostics/network-check-target-vc52k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:18:55.839556 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:55.839280 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j podName:bee553af-478a-4f02-8236-787b25adaadc nodeName:}" failed. No retries permitted until 2026-07-08 04:19:03.839261996 +0000 UTC m=+36.276036954 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpt6j" (UniqueName: "kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j") pod "network-check-target-vc52k" (UID: "bee553af-478a-4f02-8236-787b25adaadc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:18:55.939989 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:55.939922 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs\") pod \"network-metrics-daemon-6j45q\" (UID: \"37c1fce9-b855-4864-a4e0-04b78dd27b62\") " pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:18:55.939989 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:55.939977 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret\") pod \"global-pull-secret-syncer-58wwx\" (UID: \"471fcdae-c4d6-4999-b260-0f0719e99682\") " pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:55.940168 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:55.940077 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 04:18:55.940168 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:55.940132 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret podName:471fcdae-c4d6-4999-b260-0f0719e99682 nodeName:}" failed. No retries permitted until 2026-07-08 04:19:03.940115893 +0000 UTC m=+36.376890849 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret") pod "global-pull-secret-syncer-58wwx" (UID: "471fcdae-c4d6-4999-b260-0f0719e99682") : object "kube-system"/"original-pull-secret" not registered Jul 08 04:18:55.940168 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:55.940077 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:18:55.940279 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:55.940172 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs podName:37c1fce9-b855-4864-a4e0-04b78dd27b62 nodeName:}" failed. No retries permitted until 2026-07-08 04:19:03.940162612 +0000 UTC m=+36.376937569 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs") pod "network-metrics-daemon-6j45q" (UID: "37c1fce9-b855-4864-a4e0-04b78dd27b62") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:18:56.068683 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:56.068620 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:18:56.068839 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:56.068619 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:56.068839 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:56.068745 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:18:56.068839 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:56.068633 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:18:56.068839 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:56.068817 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:18:56.069053 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:56.068896 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:18:58.071491 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:58.071461 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:18:58.071947 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:58.071569 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:18:58.071947 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:58.071911 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:18:58.072065 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:58.071999 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:18:58.072065 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:58.072057 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:18:58.072161 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:18:58.072134 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:18:58.231403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:58.231080 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rvj" event={"ID":"e2812fd7-bf09-4088-b1eb-67b5ac18f380","Type":"ContainerStarted","Data":"1aaf56d40fb12f1902421d27101c024066cc1f726b9cd94bb39622929ec3da9c"} Jul 08 04:18:58.233625 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:58.233456 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4rws5" event={"ID":"5331e3cb-cabb-4cd9-b7ca-5977c7acaab7","Type":"ContainerStarted","Data":"5c4943363ded34e57085e91c07b75dd91470a26f2aa4f8df5c84ab4dc3f09a94"} Jul 08 04:18:58.236063 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:58.236034 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" event={"ID":"7b3c4c01-4894-4267-8e0d-418451b980ff","Type":"ContainerStarted","Data":"cc49a69e3a40b5dfdb10cfb233492579af7f2fae3176e752421fe903952b1bfa"} Jul 08 04:18:58.238409 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:58.238381 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-slm66" event={"ID":"03875a1c-3c96-419a-88a5-631347a6cd00","Type":"ContainerStarted","Data":"5af6a96832c5d4d2d73eba27f22b8dee8e5036aa1a069d0f4d9c86475de6e0bc"} Jul 08 04:18:58.240656 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:58.240488 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zc555" event={"ID":"b878e496-2a4c-41db-9efd-756f89197d14","Type":"ContainerStarted","Data":"fc94c4429629dcf61ccc7591ac04c92db82939e92b7d61dc130e6ec6505c7e43"} Jul 08 04:18:58.243043 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:58.242894 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m5qfn" event={"ID":"986153ab-266b-49f8-846e-cdde4d6b90af","Type":"ContainerStarted","Data":"46749b9d3362357c83dd737403aaffe77147b47549d242068302a43302ef182f"} Jul 08 04:18:58.266288 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:58.266182 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zc555" podStartSLOduration=0.687008622 podStartE2EDuration="10.266165267s" totalImagesPullingTime="9.579156645s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:18:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:58.265487644 +0000 UTC m=+30.702262650" watchObservedRunningTime="2026-07-08 04:18:58.266165267 +0000 UTC m=+30.702940248" Jul 08 04:18:58.279214 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:58.279163 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4rws5" podStartSLOduration=1.447376708 podStartE2EDuration="11.279145552s" totalImagesPullingTime="9.831768844s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:58.279022544 +0000 UTC m=+30.715797526" watchObservedRunningTime="2026-07-08 04:18:58.279145552 +0000 UTC m=+30.715920531" Jul 08 04:18:58.306537 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:58.306490 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-m5qfn" podStartSLOduration=1.572384327 podStartE2EDuration="11.306472858s" totalImagesPullingTime="9.734088531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:58.293449318 +0000 UTC m=+30.730224298" watchObservedRunningTime="2026-07-08 04:18:58.306472858 +0000 UTC m=+30.743247837" Jul 08 04:18:59.247552 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:59.247511 2583 generic.go:356] "Generic (PLEG): container finished" podID="e2812fd7-bf09-4088-b1eb-67b5ac18f380" containerID="1aaf56d40fb12f1902421d27101c024066cc1f726b9cd94bb39622929ec3da9c" exitCode=0 Jul 08 04:18:59.249743 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:59.248838 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rvj" event={"ID":"e2812fd7-bf09-4088-b1eb-67b5ac18f380","Type":"ContainerDied","Data":"1aaf56d40fb12f1902421d27101c024066cc1f726b9cd94bb39622929ec3da9c"} Jul 08 04:18:59.273280 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:18:59.273234 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-slm66" podStartSLOduration=2.519896116 podStartE2EDuration="12.273216262s" totalImagesPullingTime="9.753320146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:18:58.306964195 +0000 UTC m=+30.743739178" watchObservedRunningTime="2026-07-08 04:18:59.273216262 +0000 UTC m=+31.709991260" Jul 08 04:19:00.068082 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:00.068043 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:00.068255 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:00.068043 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:00.068321 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:00.068261 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:00.068321 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:00.068258 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:19:00.068459 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:00.068382 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:19:00.068559 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:00.068532 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:19:00.250828 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:00.250783 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-58pdq" event={"ID":"6fc503c2-f8af-433e-8450-0f18d2761150","Type":"ContainerStarted","Data":"34f666b1f28907dfe071884fe9d08eaf67f2fb202fe2e5af365ea81d82babb54"} Jul 08 04:19:00.265069 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:00.265028 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-58pdq" podStartSLOduration=3.422148768 podStartE2EDuration="13.265011942s" totalImagesPullingTime="9.842863174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:00.26463896 +0000 UTC m=+32.701413940" watchObservedRunningTime="2026-07-08 04:19:00.265011942 +0000 UTC m=+32.701786922" Jul 08 04:19:02.068174 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:02.068129 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:02.068662 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:02.068129 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:02.068662 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:02.068250 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:19:02.068662 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:02.068134 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:02.068662 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:02.068357 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:19:02.068662 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:02.068490 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:19:03.227613 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:03.227582 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-m5qfn" Jul 08 04:19:03.228233 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:03.228214 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-m5qfn" Jul 08 04:19:03.908002 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:03.907953 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpt6j\" (UniqueName: \"kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j\") pod \"network-check-target-vc52k\" (UID: \"bee553af-478a-4f02-8236-787b25adaadc\") " pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:03.908187 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:03.908113 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 04:19:03.908187 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:03.908133 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 04:19:03.908187 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:03.908144 2583 projected.go:196] Error preparing data for projected volume kube-api-access-lpt6j for pod openshift-network-diagnostics/network-check-target-vc52k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:19:03.908386 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:03.908202 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j podName:bee553af-478a-4f02-8236-787b25adaadc nodeName:}" failed. No retries permitted until 2026-07-08 04:19:19.908183304 +0000 UTC m=+52.344958262 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpt6j" (UniqueName: "kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j") pod "network-check-target-vc52k" (UID: "bee553af-478a-4f02-8236-787b25adaadc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:19:04.008594 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:04.008560 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret\") pod \"global-pull-secret-syncer-58wwx\" (UID: \"471fcdae-c4d6-4999-b260-0f0719e99682\") " pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:04.008771 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:04.008617 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs\") pod \"network-metrics-daemon-6j45q\" (UID: \"37c1fce9-b855-4864-a4e0-04b78dd27b62\") " pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:04.008771 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:04.008702 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 04:19:04.008771 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:04.008715 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:19:04.008771 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:04.008768 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs podName:37c1fce9-b855-4864-a4e0-04b78dd27b62 nodeName:}" failed. No retries permitted until 2026-07-08 04:19:20.008751773 +0000 UTC m=+52.445526735 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs") pod "network-metrics-daemon-6j45q" (UID: "37c1fce9-b855-4864-a4e0-04b78dd27b62") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:19:04.008956 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:04.008786 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret podName:471fcdae-c4d6-4999-b260-0f0719e99682 nodeName:}" failed. No retries permitted until 2026-07-08 04:19:20.008777173 +0000 UTC m=+52.445552131 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret") pod "global-pull-secret-syncer-58wwx" (UID: "471fcdae-c4d6-4999-b260-0f0719e99682") : object "kube-system"/"original-pull-secret" not registered Jul 08 04:19:04.068262 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:04.068221 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:04.068467 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:04.068385 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:19:04.068605 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:04.068581 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:04.068786 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:04.068764 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:19:04.068849 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:04.068808 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:04.068897 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:04.068864 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:19:05.058149 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.058003 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-m5qfn" Jul 08 04:19:05.058149 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.058142 2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 08 04:19:05.059081 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.058759 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m5qfn" Jul 08 04:19:05.090769 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.090737 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mfkzr"] Jul 08 04:19:05.172896 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.172866 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.175424 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.175399 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-grz7w\"" Jul 08 04:19:05.175573 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.175447 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 04:19:05.175573 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.175450 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 04:19:05.175573 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.175447 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 04:19:05.176518 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.176496 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 04:19:05.176649 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.176496 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 04:19:05.176649 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.176541 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 04:19:05.216850 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.216818 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-tls\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.216998 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.216883 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfcc5\" (UniqueName: \"kubernetes.io/projected/504a69c8-25ec-485e-ae7c-d75a74f1620d-kube-api-access-cfcc5\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.216998 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.216913 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-wtmp\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.216998 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.216937 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/504a69c8-25ec-485e-ae7c-d75a74f1620d-metrics-client-ca\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.217165 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.217006 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-textfile\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.217165 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.217090 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/504a69c8-25ec-485e-ae7c-d75a74f1620d-root\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.217165 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.217130 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-accelerators-collector-config\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.217165 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.217160 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/504a69c8-25ec-485e-ae7c-d75a74f1620d-sys\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.217442 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.217183 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.318079 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.317988 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/504a69c8-25ec-485e-ae7c-d75a74f1620d-sys\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.318079 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.318042 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.318429 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.318081 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-tls\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.318429 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.318101 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/504a69c8-25ec-485e-ae7c-d75a74f1620d-sys\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.318429 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.318126 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfcc5\" (UniqueName: \"kubernetes.io/projected/504a69c8-25ec-485e-ae7c-d75a74f1620d-kube-api-access-cfcc5\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.318429 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.318171 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-wtmp\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.318429 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.318190 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/504a69c8-25ec-485e-ae7c-d75a74f1620d-metrics-client-ca\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.318429 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.318218 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-textfile\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.318429 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:05.318226 2583 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 04:19:05.318429 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:05.318288 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-tls podName:504a69c8-25ec-485e-ae7c-d75a74f1620d nodeName:}" failed. No retries permitted until 2026-07-08 04:19:05.818266584 +0000 UTC m=+38.255041554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-tls") pod "node-exporter-mfkzr" (UID: "504a69c8-25ec-485e-ae7c-d75a74f1620d") : secret "node-exporter-tls" not found Jul 08 04:19:05.318429 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.318326 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/504a69c8-25ec-485e-ae7c-d75a74f1620d-root\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.318429 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.318382 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-accelerators-collector-config\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.318924 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.318463 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-wtmp\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.318924 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.318574 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/504a69c8-25ec-485e-ae7c-d75a74f1620d-root\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.318924 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.318802 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-textfile\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.319074 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.318980 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-accelerators-collector-config\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.319129 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.319088 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/504a69c8-25ec-485e-ae7c-d75a74f1620d-metrics-client-ca\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.322553 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.322532 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.326767 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.326748 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfcc5\" (UniqueName: \"kubernetes.io/projected/504a69c8-25ec-485e-ae7c-d75a74f1620d-kube-api-access-cfcc5\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.821444 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:05.821399 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-tls\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:05.821645 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:05.821569 2583 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 04:19:05.821719 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:05.821649 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-tls podName:504a69c8-25ec-485e-ae7c-d75a74f1620d nodeName:}" failed. No retries permitted until 2026-07-08 04:19:06.821626019 +0000 UTC m=+39.258400991 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-tls") pod "node-exporter-mfkzr" (UID: "504a69c8-25ec-485e-ae7c-d75a74f1620d") : secret "node-exporter-tls" not found Jul 08 04:19:06.068493 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:06.068457 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:06.069328 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:06.068457 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:06.069328 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:06.068607 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:19:06.069328 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:06.068458 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:06.069328 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:06.068669 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:19:06.069328 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:06.068793 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:19:06.830113 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:06.830070 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-tls\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:06.832771 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:06.832737 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/504a69c8-25ec-485e-ae7c-d75a74f1620d-node-exporter-tls\") pod \"node-exporter-mfkzr\" (UID: \"504a69c8-25ec-485e-ae7c-d75a74f1620d\") " pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:06.983317 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:06.983266 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mfkzr" Jul 08 04:19:08.071202 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:08.071172 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:08.071624 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:08.071271 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:08.071624 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:08.071288 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:19:08.071624 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:08.071339 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:19:08.071624 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:08.071366 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:08.071624 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:08.071450 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:19:09.618517 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:19:09.618466 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod504a69c8_25ec_485e_ae7c_d75a74f1620d.slice/crio-bab343af849d016d913a6c5eee71890231bac78c59abc0d7d2c39a9ee02ed9e2 WatchSource:0}: Error finding container bab343af849d016d913a6c5eee71890231bac78c59abc0d7d2c39a9ee02ed9e2: Status 404 returned error can't find the container with id bab343af849d016d913a6c5eee71890231bac78c59abc0d7d2c39a9ee02ed9e2 Jul 08 04:19:09.747924 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:09.747521 2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 04:19:09.961306 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:09.961213 2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T04:19:09.74773118Z","UUID":"7caf6f96-4717-4082-be86-e135b631fb2f","Handler":null,"Name":"","Endpoint":""} Jul 08 04:19:09.963481 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:09.963461 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 04:19:09.963583 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:09.963505 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 04:19:10.068456 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:10.068434 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:10.068561 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:10.068541 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:19:10.068654 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:10.068631 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:10.068834 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:10.068796 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:19:10.068878 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:10.068434 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:10.068915 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:10.068890 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:19:10.274689 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:10.274653 2583 generic.go:356] "Generic (PLEG): container finished" podID="e2812fd7-bf09-4088-b1eb-67b5ac18f380" containerID="8a5a6ca9d8c930b45214621e344443bb03b2ae446bf2e5d570edcb10adcbf26e" exitCode=0 Jul 08 04:19:10.274874 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:10.274742 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rvj" event={"ID":"e2812fd7-bf09-4088-b1eb-67b5ac18f380","Type":"ContainerDied","Data":"8a5a6ca9d8c930b45214621e344443bb03b2ae446bf2e5d570edcb10adcbf26e"} Jul 08 04:19:10.276375 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:10.276283 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mfkzr" event={"ID":"504a69c8-25ec-485e-ae7c-d75a74f1620d","Type":"ContainerStarted","Data":"bab343af849d016d913a6c5eee71890231bac78c59abc0d7d2c39a9ee02ed9e2"} Jul 08 04:19:10.279491 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:10.279466 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" event={"ID":"9afcd46e-0ea4-4870-98ca-6f18f177f324","Type":"ContainerStarted","Data":"3ece64955034623fc1f28c413dab167cf9f8cdd67e37ea10994a8abeb1ccbd35"} Jul 08 04:19:10.279578 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:10.279551 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" event={"ID":"9afcd46e-0ea4-4870-98ca-6f18f177f324","Type":"ContainerStarted","Data":"f729c8d7223c5efd66fdeb3e612b5bf4b775a0ddeca5bea010c375cf97adcab1"} Jul 08 04:19:10.279647 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:10.279603 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" event={"ID":"9afcd46e-0ea4-4870-98ca-6f18f177f324","Type":"ContainerStarted","Data":"4e265dc7f8d6d0248d2f4676ebc72137921322600451b3221c97f32ab9258cdb"} Jul 08 04:19:10.279700 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:10.279652 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" event={"ID":"9afcd46e-0ea4-4870-98ca-6f18f177f324","Type":"ContainerStarted","Data":"803f2465010e9f329b0323b4b352a2dc320645ae25cb052a447ad886d19aa619"} Jul 08 04:19:10.279759 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:10.279700 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" event={"ID":"9afcd46e-0ea4-4870-98ca-6f18f177f324","Type":"ContainerStarted","Data":"0ac7dbfb1b7ebef9354c3e82018b98cf6c606b883dddbeacbc0853c8fcc8224f"} Jul 08 04:19:10.279759 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:10.279747 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" event={"ID":"9afcd46e-0ea4-4870-98ca-6f18f177f324","Type":"ContainerStarted","Data":"a369224f2f6f3f3f7f0518489258154d8c9a0248cda0428cbe20ded9a3c28f95"} Jul 08 04:19:10.281811 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:10.281790 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" event={"ID":"7b3c4c01-4894-4267-8e0d-418451b980ff","Type":"ContainerStarted","Data":"b682a61dade33658f9200ef68cde232bcb23910ac409e99fd05583e4b032f17c"} Jul 08 04:19:10.283386 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:10.283336 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f545t" event={"ID":"e6763059-f532-47e0-8523-50893eb5b3db","Type":"ContainerStarted","Data":"f71288d19fce4c70f3a296117934998c85e8b75f7ec56c325d584b8846f7fd7a"} Jul 08 04:19:10.312604 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:10.312539 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-f545t" podStartSLOduration=1.8212486810000001 podStartE2EDuration="23.31252155s" totalImagesPullingTime="21.491272869s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:10.312125343 +0000 UTC m=+42.748900347" watchObservedRunningTime="2026-07-08 04:19:10.31252155 +0000 UTC m=+42.749296531" Jul 08 04:19:11.286672 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:11.286473 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mfkzr" event={"ID":"504a69c8-25ec-485e-ae7c-d75a74f1620d","Type":"ContainerStarted","Data":"79614dcc4e4ccfcccf962c1995b937a1d161fc17f9b0c248f11da43df3590106"} Jul 08 04:19:12.068895 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:12.068855 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:12.068895 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:12.068892 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:12.069139 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:12.068860 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:12.069139 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:12.068992 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:19:12.069139 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:12.069074 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:19:12.069288 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:12.069179 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:19:12.289486 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:12.289460 2583 generic.go:356] "Generic (PLEG): container finished" podID="504a69c8-25ec-485e-ae7c-d75a74f1620d" containerID="79614dcc4e4ccfcccf962c1995b937a1d161fc17f9b0c248f11da43df3590106" exitCode=0 Jul 08 04:19:12.289808 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:12.289541 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mfkzr" event={"ID":"504a69c8-25ec-485e-ae7c-d75a74f1620d","Type":"ContainerDied","Data":"79614dcc4e4ccfcccf962c1995b937a1d161fc17f9b0c248f11da43df3590106"} Jul 08 04:19:12.291482 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:12.291461 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" event={"ID":"7b3c4c01-4894-4267-8e0d-418451b980ff","Type":"ContainerStarted","Data":"94675300911a6f40383234528fd790a8bc081108df30cb083f2ec6a280a5a899"} Jul 08 04:19:12.323799 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:12.323749 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-45n67" podStartSLOduration=2.6091057749999997 podStartE2EDuration="25.32373327s" totalImagesPullingTime="22.714627495s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:18:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:12.323709824 +0000 UTC m=+44.760484808" watchObservedRunningTime="2026-07-08 04:19:12.32373327 +0000 UTC m=+44.760508250" Jul 08 04:19:13.295107 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:13.295078 2583 generic.go:356] "Generic (PLEG): container finished" podID="e2812fd7-bf09-4088-b1eb-67b5ac18f380" containerID="029c3cdfa0d1f150802b6f12c96e5d8e945c8831fbb55a1e68080fbe8953833a" exitCode=0 Jul 08 04:19:13.295549 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:13.295165 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rvj" event={"ID":"e2812fd7-bf09-4088-b1eb-67b5ac18f380","Type":"ContainerDied","Data":"029c3cdfa0d1f150802b6f12c96e5d8e945c8831fbb55a1e68080fbe8953833a"} Jul 08 04:19:13.297402 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:13.297375 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mfkzr" event={"ID":"504a69c8-25ec-485e-ae7c-d75a74f1620d","Type":"ContainerStarted","Data":"493f92f028da9ea378357d030799b78eb941349ca11ceb7d171f173f65da685b"} Jul 08 04:19:13.297514 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:13.297428 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mfkzr" event={"ID":"504a69c8-25ec-485e-ae7c-d75a74f1620d","Type":"ContainerStarted","Data":"a303bdc6a701636646d9441cd7959480b4cc38049bee026320c88a68f3ca2170"} Jul 08 04:19:13.300528 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:13.300508 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" event={"ID":"9afcd46e-0ea4-4870-98ca-6f18f177f324","Type":"ContainerStarted","Data":"78ee1b91022c3c907c5f09f467012e43cccd5085068db52dc03b47770a47a50a"} Jul 08 04:19:13.335970 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:13.335926 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mfkzr" podStartSLOduration=6.71148538 podStartE2EDuration="8.335913259s" totalImagesPullingTime="1.478538061s" totalInitContainerRuntime="145.889818ms" isStatefulPod=true podCreationTimestamp="2026-07-08 04:19:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:13.334738616 +0000 UTC m=+45.771513595" watchObservedRunningTime="2026-07-08 04:19:13.335913259 +0000 UTC m=+45.772688237" Jul 08 04:19:14.068160 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:14.068126 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:14.068371 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:14.068132 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:14.068371 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:14.068249 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:19:14.068371 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:14.068281 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:14.068534 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:14.068390 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:19:14.068534 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:14.068463 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:19:15.307657 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:15.307618 2583 generic.go:356] "Generic (PLEG): container finished" podID="e2812fd7-bf09-4088-b1eb-67b5ac18f380" containerID="43edcee5eb7cd2cf04d1ce63d65e0993d6bb0606dc979012c22459e8fcb8887d" exitCode=0 Jul 08 04:19:15.308073 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:15.307701 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rvj" event={"ID":"e2812fd7-bf09-4088-b1eb-67b5ac18f380","Type":"ContainerDied","Data":"43edcee5eb7cd2cf04d1ce63d65e0993d6bb0606dc979012c22459e8fcb8887d"} Jul 08 04:19:15.311335 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:15.311230 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" event={"ID":"9afcd46e-0ea4-4870-98ca-6f18f177f324","Type":"ContainerStarted","Data":"7252a2ed52c0effd3bc47419ff926f3ccab706b7c821c03a4ce71388d86413e8"} Jul 08 04:19:15.311521 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:15.311503 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:19:15.311588 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:15.311577 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:19:15.327819 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:15.327792 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:19:15.354981 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:15.354940 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" podStartSLOduration=6.156046769 podStartE2EDuration="27.354925404s" totalImagesPullingTime="21.198878635s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:18:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:15.354614479 +0000 UTC m=+47.791389463" watchObservedRunningTime="2026-07-08 04:19:15.354925404 +0000 UTC m=+47.791700383" Jul 08 04:19:16.068705 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:16.068505 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:16.068908 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:16.068509 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:16.068908 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:16.068509 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:16.068908 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:16.068811 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:19:16.069080 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:16.068923 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:19:16.069080 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:16.069028 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:19:16.314246 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:16.314215 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:19:16.328737 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:16.328675 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:19:16.624222 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:16.624108 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vc52k"] Jul 08 04:19:16.624464 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:16.624240 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:16.624464 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:16.624367 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:19:16.625073 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:16.625016 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6j45q"] Jul 08 04:19:16.625180 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:16.625122 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:16.625248 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:16.625225 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:19:16.625658 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:16.625627 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-58wwx"] Jul 08 04:19:16.625748 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:16.625706 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:16.625790 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:16.625770 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:19:18.069414 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:18.069390 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:18.069968 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:18.069506 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:19:18.069968 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:18.069553 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:18.069968 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:18.069612 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:19:18.069968 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:18.069648 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:18.069968 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:18.069728 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:19:19.939876 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:19.939843 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpt6j\" (UniqueName: \"kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j\") pod \"network-check-target-vc52k\" (UID: \"bee553af-478a-4f02-8236-787b25adaadc\") " pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:19.940373 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:19.940014 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 04:19:19.940373 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:19.940036 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 04:19:19.940373 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:19.940045 2583 projected.go:196] Error preparing data for projected volume kube-api-access-lpt6j for pod openshift-network-diagnostics/network-check-target-vc52k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:19:19.940373 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:19.940102 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j podName:bee553af-478a-4f02-8236-787b25adaadc nodeName:}" failed. No retries permitted until 2026-07-08 04:19:51.940083627 +0000 UTC m=+84.376858597 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpt6j" (UniqueName: "kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j") pod "network-check-target-vc52k" (UID: "bee553af-478a-4f02-8236-787b25adaadc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 04:19:20.040746 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:20.040712 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret\") pod \"global-pull-secret-syncer-58wwx\" (UID: \"471fcdae-c4d6-4999-b260-0f0719e99682\") " pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:20.040930 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:20.040791 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs\") pod \"network-metrics-daemon-6j45q\" (UID: \"37c1fce9-b855-4864-a4e0-04b78dd27b62\") " pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:20.040930 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:20.040866 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 04:19:20.040930 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:20.040900 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:19:20.041075 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:20.040945 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret podName:471fcdae-c4d6-4999-b260-0f0719e99682 nodeName:}" failed. No retries permitted until 2026-07-08 04:19:52.040923158 +0000 UTC m=+84.477698114 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret") pod "global-pull-secret-syncer-58wwx" (UID: "471fcdae-c4d6-4999-b260-0f0719e99682") : object "kube-system"/"original-pull-secret" not registered Jul 08 04:19:20.041075 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:20.040964 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs podName:37c1fce9-b855-4864-a4e0-04b78dd27b62 nodeName:}" failed. No retries permitted until 2026-07-08 04:19:52.04095718 +0000 UTC m=+84.477732137 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs") pod "network-metrics-daemon-6j45q" (UID: "37c1fce9-b855-4864-a4e0-04b78dd27b62") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 04:19:20.068685 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:20.068649 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:20.068822 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:20.068658 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:20.068822 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:20.068739 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:19:20.068822 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:20.068766 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:20.069010 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:20.068837 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:19:20.069010 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:20.068900 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:19:22.068061 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.068037 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:22.068434 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.068037 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:22.068434 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:22.068171 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j45q" podUID="37c1fce9-b855-4864-a4e0-04b78dd27b62" Jul 08 04:19:22.068434 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.068037 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:22.068434 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:22.068217 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-58wwx" podUID="471fcdae-c4d6-4999-b260-0f0719e99682" Jul 08 04:19:22.068434 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:19:22.068275 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vc52k" podUID="bee553af-478a-4f02-8236-787b25adaadc" Jul 08 04:19:22.327804 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.327706 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-10.ec2.internal" event="NodeReady" Jul 08 04:19:22.327954 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.327922 2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 04:19:22.376581 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.376543 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jf6z2"] Jul 08 04:19:22.378869 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.378838 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jf6z2" Jul 08 04:19:22.381279 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.381249 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 04:19:22.381406 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.381326 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-v9th6\"" Jul 08 04:19:22.381406 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.381326 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 04:19:22.381406 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.381364 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 04:19:22.385741 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.385720 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mgllq"] Jul 08 04:19:22.388547 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.388521 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.388712 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.388686 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jf6z2"] Jul 08 04:19:22.390989 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.390971 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 04:19:22.391178 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.391157 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6rbmt\"" Jul 08 04:19:22.391241 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.391199 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 04:19:22.391973 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.391955 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 04:19:22.392067 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.392010 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 04:19:22.398471 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.398451 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mgllq"] Jul 08 04:19:22.460867 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.460838 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/558b5c81-e1c2-4914-8395-d1fe61e9b0a4-crio-socket\") pod \"insights-runtime-extractor-mgllq\" (UID: \"558b5c81-e1c2-4914-8395-d1fe61e9b0a4\") " pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.461035 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.460883 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcc7d\" (UniqueName: \"kubernetes.io/projected/558b5c81-e1c2-4914-8395-d1fe61e9b0a4-kube-api-access-lcc7d\") pod \"insights-runtime-extractor-mgllq\" (UID: \"558b5c81-e1c2-4914-8395-d1fe61e9b0a4\") " pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.461035 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.460947 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acdc3d88-e479-4cf3-9fe1-31fd5c1b4554-cert\") pod \"ingress-canary-jf6z2\" (UID: \"acdc3d88-e479-4cf3-9fe1-31fd5c1b4554\") " pod="openshift-ingress-canary/ingress-canary-jf6z2" Jul 08 04:19:22.461129 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.461031 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgxpj\" (UniqueName: \"kubernetes.io/projected/acdc3d88-e479-4cf3-9fe1-31fd5c1b4554-kube-api-access-dgxpj\") pod \"ingress-canary-jf6z2\" (UID: \"acdc3d88-e479-4cf3-9fe1-31fd5c1b4554\") " pod="openshift-ingress-canary/ingress-canary-jf6z2" Jul 08 04:19:22.461129 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.461063 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/558b5c81-e1c2-4914-8395-d1fe61e9b0a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgllq\" (UID: \"558b5c81-e1c2-4914-8395-d1fe61e9b0a4\") " pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.461129 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.461100 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/558b5c81-e1c2-4914-8395-d1fe61e9b0a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgllq\" (UID: \"558b5c81-e1c2-4914-8395-d1fe61e9b0a4\") " pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.461129 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.461118 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/558b5c81-e1c2-4914-8395-d1fe61e9b0a4-data-volume\") pod \"insights-runtime-extractor-mgllq\" (UID: \"558b5c81-e1c2-4914-8395-d1fe61e9b0a4\") " pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.476258 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.476228 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8mn4h"] Jul 08 04:19:22.478788 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.478771 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8mn4h" Jul 08 04:19:22.481418 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.481400 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 04:19:22.481418 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.481337 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xh8sv\"" Jul 08 04:19:22.481532 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.481458 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 04:19:22.491415 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.491395 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8mn4h"] Jul 08 04:19:22.562367 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.562320 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/558b5c81-e1c2-4914-8395-d1fe61e9b0a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgllq\" (UID: \"558b5c81-e1c2-4914-8395-d1fe61e9b0a4\") " pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.562495 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.562461 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/558b5c81-e1c2-4914-8395-d1fe61e9b0a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgllq\" (UID: \"558b5c81-e1c2-4914-8395-d1fe61e9b0a4\") " pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.562495 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.562485 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/558b5c81-e1c2-4914-8395-d1fe61e9b0a4-data-volume\") pod \"insights-runtime-extractor-mgllq\" (UID: \"558b5c81-e1c2-4914-8395-d1fe61e9b0a4\") " pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.562579 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.562505 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwdxv\" (UniqueName: \"kubernetes.io/projected/e6ad9554-4baa-4b5c-8e75-b6099a656f0c-kube-api-access-wwdxv\") pod \"dns-default-8mn4h\" (UID: \"e6ad9554-4baa-4b5c-8e75-b6099a656f0c\") " pod="openshift-dns/dns-default-8mn4h" Jul 08 04:19:22.562579 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.562543 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/558b5c81-e1c2-4914-8395-d1fe61e9b0a4-crio-socket\") pod \"insights-runtime-extractor-mgllq\" (UID: \"558b5c81-e1c2-4914-8395-d1fe61e9b0a4\") " pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.562579 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.562566 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcc7d\" (UniqueName: \"kubernetes.io/projected/558b5c81-e1c2-4914-8395-d1fe61e9b0a4-kube-api-access-lcc7d\") pod \"insights-runtime-extractor-mgllq\" (UID: \"558b5c81-e1c2-4914-8395-d1fe61e9b0a4\") " pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.562704 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.562584 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e6ad9554-4baa-4b5c-8e75-b6099a656f0c-config-volume\") pod \"dns-default-8mn4h\" (UID: \"e6ad9554-4baa-4b5c-8e75-b6099a656f0c\") " pod="openshift-dns/dns-default-8mn4h" Jul 08 04:19:22.562704 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.562599 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6ad9554-4baa-4b5c-8e75-b6099a656f0c-tmp-dir\") pod \"dns-default-8mn4h\" (UID: \"e6ad9554-4baa-4b5c-8e75-b6099a656f0c\") " pod="openshift-dns/dns-default-8mn4h" Jul 08 04:19:22.562704 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.562623 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acdc3d88-e479-4cf3-9fe1-31fd5c1b4554-cert\") pod \"ingress-canary-jf6z2\" (UID: \"acdc3d88-e479-4cf3-9fe1-31fd5c1b4554\") " pod="openshift-ingress-canary/ingress-canary-jf6z2" Jul 08 04:19:22.562704 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.562659 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6ad9554-4baa-4b5c-8e75-b6099a656f0c-metrics-tls\") pod \"dns-default-8mn4h\" (UID: \"e6ad9554-4baa-4b5c-8e75-b6099a656f0c\") " pod="openshift-dns/dns-default-8mn4h" Jul 08 04:19:22.562875 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.562723 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgxpj\" (UniqueName: \"kubernetes.io/projected/acdc3d88-e479-4cf3-9fe1-31fd5c1b4554-kube-api-access-dgxpj\") pod \"ingress-canary-jf6z2\" (UID: \"acdc3d88-e479-4cf3-9fe1-31fd5c1b4554\") " pod="openshift-ingress-canary/ingress-canary-jf6z2" Jul 08 04:19:22.562875 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.562783 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/558b5c81-e1c2-4914-8395-d1fe61e9b0a4-crio-socket\") pod \"insights-runtime-extractor-mgllq\" (UID: \"558b5c81-e1c2-4914-8395-d1fe61e9b0a4\") " pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.562875 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.562818 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/558b5c81-e1c2-4914-8395-d1fe61e9b0a4-data-volume\") pod \"insights-runtime-extractor-mgllq\" (UID: \"558b5c81-e1c2-4914-8395-d1fe61e9b0a4\") " pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.563164 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.563139 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/558b5c81-e1c2-4914-8395-d1fe61e9b0a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgllq\" (UID: \"558b5c81-e1c2-4914-8395-d1fe61e9b0a4\") " pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.565872 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.565732 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/558b5c81-e1c2-4914-8395-d1fe61e9b0a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgllq\" (UID: \"558b5c81-e1c2-4914-8395-d1fe61e9b0a4\") " pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.565966 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.565777 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acdc3d88-e479-4cf3-9fe1-31fd5c1b4554-cert\") pod \"ingress-canary-jf6z2\" (UID: \"acdc3d88-e479-4cf3-9fe1-31fd5c1b4554\") " pod="openshift-ingress-canary/ingress-canary-jf6z2" Jul 08 04:19:22.570417 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.570389 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcc7d\" (UniqueName: \"kubernetes.io/projected/558b5c81-e1c2-4914-8395-d1fe61e9b0a4-kube-api-access-lcc7d\") pod \"insights-runtime-extractor-mgllq\" (UID: \"558b5c81-e1c2-4914-8395-d1fe61e9b0a4\") " pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.570502 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.570445 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgxpj\" (UniqueName: \"kubernetes.io/projected/acdc3d88-e479-4cf3-9fe1-31fd5c1b4554-kube-api-access-dgxpj\") pod \"ingress-canary-jf6z2\" (UID: \"acdc3d88-e479-4cf3-9fe1-31fd5c1b4554\") " pod="openshift-ingress-canary/ingress-canary-jf6z2" Jul 08 04:19:22.663543 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.663508 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e6ad9554-4baa-4b5c-8e75-b6099a656f0c-config-volume\") pod \"dns-default-8mn4h\" (UID: \"e6ad9554-4baa-4b5c-8e75-b6099a656f0c\") " pod="openshift-dns/dns-default-8mn4h" Jul 08 04:19:22.663543 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.663542 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6ad9554-4baa-4b5c-8e75-b6099a656f0c-tmp-dir\") pod \"dns-default-8mn4h\" (UID: \"e6ad9554-4baa-4b5c-8e75-b6099a656f0c\") " pod="openshift-dns/dns-default-8mn4h" Jul 08 04:19:22.663751 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.663560 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6ad9554-4baa-4b5c-8e75-b6099a656f0c-metrics-tls\") pod \"dns-default-8mn4h\" (UID: \"e6ad9554-4baa-4b5c-8e75-b6099a656f0c\") " pod="openshift-dns/dns-default-8mn4h" Jul 08 04:19:22.663751 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.663619 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwdxv\" (UniqueName: \"kubernetes.io/projected/e6ad9554-4baa-4b5c-8e75-b6099a656f0c-kube-api-access-wwdxv\") pod \"dns-default-8mn4h\" (UID: \"e6ad9554-4baa-4b5c-8e75-b6099a656f0c\") " pod="openshift-dns/dns-default-8mn4h" Jul 08 04:19:22.663922 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.663899 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6ad9554-4baa-4b5c-8e75-b6099a656f0c-tmp-dir\") pod \"dns-default-8mn4h\" (UID: \"e6ad9554-4baa-4b5c-8e75-b6099a656f0c\") " pod="openshift-dns/dns-default-8mn4h" Jul 08 04:19:22.664124 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.664100 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e6ad9554-4baa-4b5c-8e75-b6099a656f0c-config-volume\") pod \"dns-default-8mn4h\" (UID: \"e6ad9554-4baa-4b5c-8e75-b6099a656f0c\") " pod="openshift-dns/dns-default-8mn4h" Jul 08 04:19:22.666328 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.666309 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6ad9554-4baa-4b5c-8e75-b6099a656f0c-metrics-tls\") pod \"dns-default-8mn4h\" (UID: \"e6ad9554-4baa-4b5c-8e75-b6099a656f0c\") " pod="openshift-dns/dns-default-8mn4h" Jul 08 04:19:22.671181 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.671162 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwdxv\" (UniqueName: \"kubernetes.io/projected/e6ad9554-4baa-4b5c-8e75-b6099a656f0c-kube-api-access-wwdxv\") pod \"dns-default-8mn4h\" (UID: \"e6ad9554-4baa-4b5c-8e75-b6099a656f0c\") " pod="openshift-dns/dns-default-8mn4h" Jul 08 04:19:22.691002 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.690971 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jf6z2" Jul 08 04:19:22.698766 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.698749 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mgllq" Jul 08 04:19:22.788812 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.788760 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8mn4h" Jul 08 04:19:22.826257 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:22.826228 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jf6z2"] Jul 08 04:19:24.069195 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:24.069137 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:24.070014 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:24.069151 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:24.070014 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:24.069262 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:24.072092 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:24.072063 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bbjls\"" Jul 08 04:19:24.073227 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:24.073186 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 04:19:24.073227 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:24.073189 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 04:19:24.076823 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:24.073286 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 04:19:24.076823 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:24.073857 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gp6ks\"" Jul 08 04:19:24.076823 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:24.074038 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 04:19:24.211439 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:19:24.211404 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacdc3d88_e479_4cf3_9fe1_31fd5c1b4554.slice/crio-80de637e93a034f226aa6ddbfce20ecf7a2c0c07b3a491440253b98b69891881 WatchSource:0}: Error finding container 80de637e93a034f226aa6ddbfce20ecf7a2c0c07b3a491440253b98b69891881: Status 404 returned error can't find the container with id 80de637e93a034f226aa6ddbfce20ecf7a2c0c07b3a491440253b98b69891881 Jul 08 04:19:24.334295 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:24.334212 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jf6z2" event={"ID":"acdc3d88-e479-4cf3-9fe1-31fd5c1b4554","Type":"ContainerStarted","Data":"80de637e93a034f226aa6ddbfce20ecf7a2c0c07b3a491440253b98b69891881"} Jul 08 04:19:24.352992 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:24.352925 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mgllq"] Jul 08 04:19:24.355449 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:24.355424 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8mn4h"] Jul 08 04:19:24.356696 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:19:24.356674 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod558b5c81_e1c2_4914_8395_d1fe61e9b0a4.slice/crio-598f27b2065cd9a034bda0c5be716b58d88688e244c240fe264cbce39e910510 WatchSource:0}: Error finding container 598f27b2065cd9a034bda0c5be716b58d88688e244c240fe264cbce39e910510: Status 404 returned error can't find the container with id 598f27b2065cd9a034bda0c5be716b58d88688e244c240fe264cbce39e910510 Jul 08 04:19:24.358507 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:19:24.358469 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6ad9554_4baa_4b5c_8e75_b6099a656f0c.slice/crio-9ae3b51acd9541fb8e8fcf286e18423e5cc373fbf0e60e76c3a48dd90f66b153 WatchSource:0}: Error finding container 9ae3b51acd9541fb8e8fcf286e18423e5cc373fbf0e60e76c3a48dd90f66b153: Status 404 returned error can't find the container with id 9ae3b51acd9541fb8e8fcf286e18423e5cc373fbf0e60e76c3a48dd90f66b153 Jul 08 04:19:25.340128 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:25.339840 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8mn4h" event={"ID":"e6ad9554-4baa-4b5c-8e75-b6099a656f0c","Type":"ContainerStarted","Data":"9ae3b51acd9541fb8e8fcf286e18423e5cc373fbf0e60e76c3a48dd90f66b153"} Jul 08 04:19:25.344269 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:25.343364 2583 generic.go:356] "Generic (PLEG): container finished" podID="e2812fd7-bf09-4088-b1eb-67b5ac18f380" containerID="41b3b960823637b355f9f40fe79872681c1f53cea51fb302f013e7ae5fbfc007" exitCode=0 Jul 08 04:19:25.344269 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:25.343446 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rvj" event={"ID":"e2812fd7-bf09-4088-b1eb-67b5ac18f380","Type":"ContainerDied","Data":"41b3b960823637b355f9f40fe79872681c1f53cea51fb302f013e7ae5fbfc007"} Jul 08 04:19:25.346476 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:25.346422 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mgllq" event={"ID":"558b5c81-e1c2-4914-8395-d1fe61e9b0a4","Type":"ContainerStarted","Data":"598f27b2065cd9a034bda0c5be716b58d88688e244c240fe264cbce39e910510"} Jul 08 04:19:26.353011 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:26.352977 2583 generic.go:356] "Generic (PLEG): container finished" podID="e2812fd7-bf09-4088-b1eb-67b5ac18f380" containerID="957299a52af543a2c0c13ff16cb0e0581b93d9d0f93edb1de1f14e35f9120e82" exitCode=0 Jul 08 04:19:26.353401 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:26.353026 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rvj" event={"ID":"e2812fd7-bf09-4088-b1eb-67b5ac18f380","Type":"ContainerDied","Data":"957299a52af543a2c0c13ff16cb0e0581b93d9d0f93edb1de1f14e35f9120e82"} Jul 08 04:19:30.371744 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:30.371470 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s4rvj" event={"ID":"e2812fd7-bf09-4088-b1eb-67b5ac18f380","Type":"ContainerStarted","Data":"b4cbbd63dc021c453641cdba662c0542390cd00d5db94be7fea2b396f64fe2b4"} Jul 08 04:19:30.373043 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:30.373017 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mgllq" event={"ID":"558b5c81-e1c2-4914-8395-d1fe61e9b0a4","Type":"ContainerStarted","Data":"79e8d04564c5854baf27c7efa5d3f1fd12855be5270abe717dd9038685a136d7"} Jul 08 04:19:30.374591 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:30.374561 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jf6z2" event={"ID":"acdc3d88-e479-4cf3-9fe1-31fd5c1b4554","Type":"ContainerStarted","Data":"1569224973cc3473294b193464e1d06148c87561182d1c551c0387824397ad98"} Jul 08 04:19:30.376443 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:30.376415 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8mn4h" event={"ID":"e6ad9554-4baa-4b5c-8e75-b6099a656f0c","Type":"ContainerStarted","Data":"dbb0b2d84d4297b3b8435396e5ab4637d3f960828c0334a6f0256373a6115060"} Jul 08 04:19:30.376559 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:30.376463 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8mn4h" event={"ID":"e6ad9554-4baa-4b5c-8e75-b6099a656f0c","Type":"ContainerStarted","Data":"ccc52f5caf054a7cc0f4700ddf53c4030fb58ddcafad941165bee3d77cf9ff8e"} Jul 08 04:19:30.376617 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:30.376594 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8mn4h" Jul 08 04:19:30.394667 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:30.394613 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-s4rvj" podStartSLOduration=11.03714194 podStartE2EDuration="43.394598346s" totalImagesPullingTime="32.357456406s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:18:47 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:30.393285485 +0000 UTC m=+62.830060464" watchObservedRunningTime="2026-07-08 04:19:30.394598346 +0000 UTC m=+62.831373324" Jul 08 04:19:30.411051 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:30.410996 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jf6z2" podStartSLOduration=2.989483186 podStartE2EDuration="8.410978631s" totalImagesPullingTime="5.421495445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:30.41013079 +0000 UTC m=+62.846905764" watchObservedRunningTime="2026-07-08 04:19:30.410978631 +0000 UTC m=+62.847753587" Jul 08 04:19:30.426551 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:30.426492 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8mn4h" podStartSLOduration=3.1526670709999998 podStartE2EDuration="8.426474713s" totalImagesPullingTime="5.273807642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:30.425806887 +0000 UTC m=+62.862581867" watchObservedRunningTime="2026-07-08 04:19:30.426474713 +0000 UTC m=+62.863249691" Jul 08 04:19:31.383157 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:31.383112 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mgllq" event={"ID":"558b5c81-e1c2-4914-8395-d1fe61e9b0a4","Type":"ContainerStarted","Data":"fc076ba37e341a20171879a9eaf64946f503a407ec60ad2a0068886b0232f771"} Jul 08 04:19:34.395717 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:34.395680 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mgllq" event={"ID":"558b5c81-e1c2-4914-8395-d1fe61e9b0a4","Type":"ContainerStarted","Data":"8216e1dbff9f07a9144c5b90e6bf40c97d8553828befebb464368c83f9a6a969"} Jul 08 04:19:34.413641 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:34.413600 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mgllq" podStartSLOduration=3.24943511 podStartE2EDuration="12.413585645s" totalImagesPullingTime="9.164150535s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:19:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:34.412855006 +0000 UTC m=+66.849629985" watchObservedRunningTime="2026-07-08 04:19:34.413585645 +0000 UTC m=+66.850360623" Jul 08 04:19:40.385165 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:40.385132 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8mn4h" Jul 08 04:19:48.328015 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:48.327988 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qthv2" Jul 08 04:19:51.992085 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:51.991971 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpt6j\" (UniqueName: \"kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j\") pod \"network-check-target-vc52k\" (UID: \"bee553af-478a-4f02-8236-787b25adaadc\") " pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:51.994797 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:51.994779 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 04:19:52.004830 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.004812 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 04:19:52.016565 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.016544 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpt6j\" (UniqueName: \"kubernetes.io/projected/bee553af-478a-4f02-8236-787b25adaadc-kube-api-access-lpt6j\") pod \"network-check-target-vc52k\" (UID: \"bee553af-478a-4f02-8236-787b25adaadc\") " pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:52.092734 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.092708 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs\") pod \"network-metrics-daemon-6j45q\" (UID: \"37c1fce9-b855-4864-a4e0-04b78dd27b62\") " pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:52.092905 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.092752 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret\") pod \"global-pull-secret-syncer-58wwx\" (UID: \"471fcdae-c4d6-4999-b260-0f0719e99682\") " pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:52.095210 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.095195 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 04:19:52.095267 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.095196 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 04:19:52.105193 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.105176 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/471fcdae-c4d6-4999-b260-0f0719e99682-original-pull-secret\") pod \"global-pull-secret-syncer-58wwx\" (UID: \"471fcdae-c4d6-4999-b260-0f0719e99682\") " pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:52.105280 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.105196 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37c1fce9-b855-4864-a4e0-04b78dd27b62-metrics-certs\") pod \"network-metrics-daemon-6j45q\" (UID: \"37c1fce9-b855-4864-a4e0-04b78dd27b62\") " pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:52.284736 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.284684 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bbjls\"" Jul 08 04:19:52.289505 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.289491 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gp6ks\"" Jul 08 04:19:52.293420 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.293404 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:52.293420 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.293417 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-58wwx" Jul 08 04:19:52.298006 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.297994 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j45q" Jul 08 04:19:52.434668 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.434642 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vc52k"] Jul 08 04:19:52.437173 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:19:52.437143 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbee553af_478a_4f02_8236_787b25adaadc.slice/crio-6dcb04f915222de45abcaa8d475d2a35acf1524c602b0ed376d0aebfde3c0ba7 WatchSource:0}: Error finding container 6dcb04f915222de45abcaa8d475d2a35acf1524c602b0ed376d0aebfde3c0ba7: Status 404 returned error can't find the container with id 6dcb04f915222de45abcaa8d475d2a35acf1524c602b0ed376d0aebfde3c0ba7 Jul 08 04:19:52.449357 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.449321 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vc52k" event={"ID":"bee553af-478a-4f02-8236-787b25adaadc","Type":"ContainerStarted","Data":"6dcb04f915222de45abcaa8d475d2a35acf1524c602b0ed376d0aebfde3c0ba7"} Jul 08 04:19:52.461872 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.461857 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jf6z2_acdc3d88-e479-4cf3-9fe1-31fd5c1b4554/serve-healthcheck-canary/0.log" Jul 08 04:19:52.665129 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.665105 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-58wwx"] Jul 08 04:19:52.667700 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:52.667676 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6j45q"] Jul 08 04:19:52.668432 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:19:52.668411 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod471fcdae_c4d6_4999_b260_0f0719e99682.slice/crio-0cb0368123660142fcf1a58e39d0597b87159bad7b895e02ef98e04c0dc592f1 WatchSource:0}: Error finding container 0cb0368123660142fcf1a58e39d0597b87159bad7b895e02ef98e04c0dc592f1: Status 404 returned error can't find the container with id 0cb0368123660142fcf1a58e39d0597b87159bad7b895e02ef98e04c0dc592f1 Jul 08 04:19:52.670001 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:19:52.669974 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37c1fce9_b855_4864_a4e0_04b78dd27b62.slice/crio-45ce197b63a24cd2c29857eb892cb5cfc6b1c97157131414436b7effb412db53 WatchSource:0}: Error finding container 45ce197b63a24cd2c29857eb892cb5cfc6b1c97157131414436b7effb412db53: Status 404 returned error can't find the container with id 45ce197b63a24cd2c29857eb892cb5cfc6b1c97157131414436b7effb412db53 Jul 08 04:19:53.453938 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:53.453862 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-58wwx" event={"ID":"471fcdae-c4d6-4999-b260-0f0719e99682","Type":"ContainerStarted","Data":"0cb0368123660142fcf1a58e39d0597b87159bad7b895e02ef98e04c0dc592f1"} Jul 08 04:19:53.455411 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:53.455386 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6j45q" event={"ID":"37c1fce9-b855-4864-a4e0-04b78dd27b62","Type":"ContainerStarted","Data":"45ce197b63a24cd2c29857eb892cb5cfc6b1c97157131414436b7effb412db53"} Jul 08 04:19:54.461456 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:54.461423 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6j45q" event={"ID":"37c1fce9-b855-4864-a4e0-04b78dd27b62","Type":"ContainerStarted","Data":"f0b69c740632ff7db48851f00850b650d7fef72eda2048cf5a18b403ff53925b"} Jul 08 04:19:54.461872 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:54.461480 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6j45q" event={"ID":"37c1fce9-b855-4864-a4e0-04b78dd27b62","Type":"ContainerStarted","Data":"053bd2f0e7c95c0570788df1d0909d906a157580bce00ff3c424efcf849f2be2"} Jul 08 04:19:54.477516 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:54.477456 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6j45q" podStartSLOduration=65.217145507 podStartE2EDuration="1m6.477438195s" totalImagesPullingTime="1.260292688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:18:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:54.477204751 +0000 UTC m=+86.913979732" watchObservedRunningTime="2026-07-08 04:19:54.477438195 +0000 UTC m=+86.914213176" Jul 08 04:19:56.470627 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:56.470588 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vc52k" event={"ID":"bee553af-478a-4f02-8236-787b25adaadc","Type":"ContainerStarted","Data":"890d2bdd0548f8c6f776ab6c885cf4fd7ba39a702d86bd0818d4aabdf26d5fa6"} Jul 08 04:19:56.471058 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:56.470726 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:19:56.486085 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:56.486041 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vc52k" podStartSLOduration=65.840317104 podStartE2EDuration="1m9.486024578s" totalImagesPullingTime="3.645707474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:56.484885334 +0000 UTC m=+88.921660314" watchObservedRunningTime="2026-07-08 04:19:56.486024578 +0000 UTC m=+88.922799558" Jul 08 04:19:58.479182 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:58.479144 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-58wwx" event={"ID":"471fcdae-c4d6-4999-b260-0f0719e99682","Type":"ContainerStarted","Data":"d72b0e0531cc980c8d6fbfdab1c8568cf609f56f48418da44d2c4d45ee7b8b71"} Jul 08 04:19:58.495240 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:19:58.495188 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-58wwx" podStartSLOduration=65.229291516 podStartE2EDuration="1m10.495171968s" totalImagesPullingTime="5.265880452s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:18:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:19:58.494620954 +0000 UTC m=+90.931395948" watchObservedRunningTime="2026-07-08 04:19:58.495171968 +0000 UTC m=+90.931946946" Jul 08 04:20:27.476651 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:27.476622 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vc52k" Jul 08 04:20:29.338003 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.337969 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-c79c98b48-lrbzw"] Jul 08 04:20:29.367963 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.367937 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c79c98b48-lrbzw"] Jul 08 04:20:29.368101 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.368038 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.370487 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.370458 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 04:20:29.370620 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.370537 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 04:20:29.370620 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.370561 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-gsxkc\"" Jul 08 04:20:29.370620 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.370569 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 04:20:29.370869 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.370855 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 04:20:29.370913 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.370894 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 04:20:29.376514 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.376491 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 04:20:29.446290 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.446259 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5c80577e-8424-4730-9d52-a621396c324c-telemeter-client-tls\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.446290 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.446290 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5c80577e-8424-4730-9d52-a621396c324c-secret-telemeter-client\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.446467 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.446307 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c80577e-8424-4730-9d52-a621396c324c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.446467 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.446339 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c80577e-8424-4730-9d52-a621396c324c-serving-certs-ca-bundle\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.446467 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.446375 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5c80577e-8424-4730-9d52-a621396c324c-federate-client-tls\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.446467 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.446390 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c80577e-8424-4730-9d52-a621396c324c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.446467 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.446409 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbh4k\" (UniqueName: \"kubernetes.io/projected/5c80577e-8424-4730-9d52-a621396c324c-kube-api-access-bbh4k\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.446642 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.446512 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c80577e-8424-4730-9d52-a621396c324c-metrics-client-ca\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.547568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.547533 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5c80577e-8424-4730-9d52-a621396c324c-telemeter-client-tls\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.547722 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.547578 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5c80577e-8424-4730-9d52-a621396c324c-secret-telemeter-client\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.547722 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.547629 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c80577e-8424-4730-9d52-a621396c324c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.547722 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.547669 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c80577e-8424-4730-9d52-a621396c324c-serving-certs-ca-bundle\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.547722 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.547687 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5c80577e-8424-4730-9d52-a621396c324c-federate-client-tls\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.547722 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.547707 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c80577e-8424-4730-9d52-a621396c324c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.547962 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.547725 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbh4k\" (UniqueName: \"kubernetes.io/projected/5c80577e-8424-4730-9d52-a621396c324c-kube-api-access-bbh4k\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.547962 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.547768 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c80577e-8424-4730-9d52-a621396c324c-metrics-client-ca\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.548910 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.548885 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c80577e-8424-4730-9d52-a621396c324c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.549030 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.548910 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c80577e-8424-4730-9d52-a621396c324c-metrics-client-ca\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.549206 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.549188 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c80577e-8424-4730-9d52-a621396c324c-serving-certs-ca-bundle\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.550160 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.550132 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5c80577e-8424-4730-9d52-a621396c324c-telemeter-client-tls\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.550549 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.550531 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5c80577e-8424-4730-9d52-a621396c324c-federate-client-tls\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.550645 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.550604 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5c80577e-8424-4730-9d52-a621396c324c-secret-telemeter-client\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.550645 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.550632 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c80577e-8424-4730-9d52-a621396c324c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.555751 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.555731 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbh4k\" (UniqueName: \"kubernetes.io/projected/5c80577e-8424-4730-9d52-a621396c324c-kube-api-access-bbh4k\") pod \"telemeter-client-c79c98b48-lrbzw\" (UID: \"5c80577e-8424-4730-9d52-a621396c324c\") " pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.677613 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.677585 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" Jul 08 04:20:29.840787 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:29.840748 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c79c98b48-lrbzw"] Jul 08 04:20:29.844363 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:20:29.844312 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c80577e_8424_4730_9d52_a621396c324c.slice/crio-be56e38df27bf2875dd092ee8c58d724e29d23745d8e1a1dcf72c0b2259720c9 WatchSource:0}: Error finding container be56e38df27bf2875dd092ee8c58d724e29d23745d8e1a1dcf72c0b2259720c9: Status 404 returned error can't find the container with id be56e38df27bf2875dd092ee8c58d724e29d23745d8e1a1dcf72c0b2259720c9 Jul 08 04:20:30.579042 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:30.579004 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" event={"ID":"5c80577e-8424-4730-9d52-a621396c324c","Type":"ContainerStarted","Data":"be56e38df27bf2875dd092ee8c58d724e29d23745d8e1a1dcf72c0b2259720c9"} Jul 08 04:20:31.764565 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.764523 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 04:20:31.768729 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.768700 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.771111 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.771083 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 04:20:31.771265 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.771122 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 04:20:31.771265 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.771219 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 04:20:31.773149 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.772493 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zdthz\"" Jul 08 04:20:31.773149 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.772667 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 04:20:31.774014 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.773561 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ab09pqc7mv9e6\"" Jul 08 04:20:31.774014 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.773867 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 04:20:31.774268 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.774152 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 04:20:31.776532 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.774374 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 04:20:31.776532 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.774540 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 04:20:31.776532 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.774711 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 04:20:31.776532 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.774860 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 04:20:31.776532 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.775031 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 04:20:31.776532 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.776503 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 04:20:31.777696 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.777547 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 04:20:31.778827 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.778809 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 04:20:31.782606 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.782540 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 04:20:31.783138 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.783110 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 04:20:31.865218 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865180 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865218 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865221 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865489 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865250 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-config\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865489 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865289 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865489 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865330 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865489 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865372 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865489 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865394 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bbb8689d-8508-4493-9021-34220781027e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865489 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865416 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865489 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865443 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865883 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865528 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865883 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865603 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865883 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865633 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865883 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865667 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865883 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865697 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbb8689d-8508-4493-9021-34220781027e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865883 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865721 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-web-config\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865883 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865753 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865883 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865808 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbb8689d-8508-4493-9021-34220781027e-config-out\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.865883 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865842 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2fd6\" (UniqueName: \"kubernetes.io/projected/bbb8689d-8508-4493-9021-34220781027e-kube-api-access-p2fd6\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.866181 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865871 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.866181 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.865914 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967135 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967095 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967135 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967139 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967386 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967187 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967386 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967240 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbb8689d-8508-4493-9021-34220781027e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967386 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967266 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-web-config\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967386 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967322 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967390 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbb8689d-8508-4493-9021-34220781027e-config-out\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967429 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2fd6\" (UniqueName: \"kubernetes.io/projected/bbb8689d-8508-4493-9021-34220781027e-kube-api-access-p2fd6\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967456 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967483 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967523 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967545 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967568 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-config\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967847 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967596 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967847 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967634 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967847 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967667 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967847 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967690 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bbb8689d-8508-4493-9021-34220781027e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967847 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967714 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967847 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967748 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.967847 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967780 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.968114 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.967984 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.968114 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.968025 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.970425 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.968671 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.970425 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.969336 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.971254 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.970917 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bbb8689d-8508-4493-9021-34220781027e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.971610 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.971405 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.971610 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.971449 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbb8689d-8508-4493-9021-34220781027e-config-out\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.971610 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.971473 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.972423 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.972185 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.972944 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.972905 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-web-config\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.973233 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.973175 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.973766 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.973667 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbb8689d-8508-4493-9021-34220781027e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.973766 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.973667 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.974580 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.974479 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.974686 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.974589 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.975221 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.975193 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-config\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.975450 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.975422 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.975603 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.975581 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bbb8689d-8508-4493-9021-34220781027e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.977034 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.977011 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2fd6\" (UniqueName: \"kubernetes.io/projected/bbb8689d-8508-4493-9021-34220781027e-kube-api-access-p2fd6\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:31.977034 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:31.977022 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bbb8689d-8508-4493-9021-34220781027e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bbb8689d-8508-4493-9021-34220781027e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:32.085702 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:32.085621 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:20:32.250194 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:32.250166 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 04:20:32.252279 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:20:32.252249 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbb8689d_8508_4493_9021_34220781027e.slice/crio-1d14c8a76be1a679b56fa8cff020d4766ad74bd1b00336d75399024ed151b8d0 WatchSource:0}: Error finding container 1d14c8a76be1a679b56fa8cff020d4766ad74bd1b00336d75399024ed151b8d0: Status 404 returned error can't find the container with id 1d14c8a76be1a679b56fa8cff020d4766ad74bd1b00336d75399024ed151b8d0 Jul 08 04:20:32.585795 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:32.585756 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbb8689d-8508-4493-9021-34220781027e","Type":"ContainerStarted","Data":"1d14c8a76be1a679b56fa8cff020d4766ad74bd1b00336d75399024ed151b8d0"} Jul 08 04:20:32.587020 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:32.586995 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" event={"ID":"5c80577e-8424-4730-9d52-a621396c324c","Type":"ContainerStarted","Data":"df1ff7763dd7bd8cde0af43212fddd4c25cd747767a842fc0ad0441b94c014db"} Jul 08 04:20:33.593451 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:33.593332 2583 generic.go:356] "Generic (PLEG): container finished" podID="bbb8689d-8508-4493-9021-34220781027e" containerID="2858d9760ad06b9ec38a87954fe7e7a3554ff625462f6446ab28e63161017221" exitCode=0 Jul 08 04:20:33.593883 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:33.593470 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbb8689d-8508-4493-9021-34220781027e","Type":"ContainerDied","Data":"2858d9760ad06b9ec38a87954fe7e7a3554ff625462f6446ab28e63161017221"} Jul 08 04:20:33.596340 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:33.596316 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" event={"ID":"5c80577e-8424-4730-9d52-a621396c324c","Type":"ContainerStarted","Data":"cf076d2cb9ded749aa7e9f51fe794f3c539f386668e62e5fefb7ed6ec3ec09ea"} Jul 08 04:20:33.596445 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:33.596385 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" event={"ID":"5c80577e-8424-4730-9d52-a621396c324c","Type":"ContainerStarted","Data":"a060a89868f43dbccf6e4354c06fa50debe32dea1e73572b009bf9752e1d4fa6"} Jul 08 04:20:33.641023 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:33.640979 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-c79c98b48-lrbzw" podStartSLOduration=1.411727119 podStartE2EDuration="4.640964579s" totalImagesPullingTime="3.22923746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:20:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:20:33.639761308 +0000 UTC m=+126.076536287" watchObservedRunningTime="2026-07-08 04:20:33.640964579 +0000 UTC m=+126.077739558" Jul 08 04:20:37.612080 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:37.612044 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbb8689d-8508-4493-9021-34220781027e","Type":"ContainerStarted","Data":"7f8cf8528beff2471c6dbf2d0c6681884f79ad076eff06ab806dd26d4c0d6905"} Jul 08 04:20:37.612553 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:37.612109 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbb8689d-8508-4493-9021-34220781027e","Type":"ContainerStarted","Data":"6daf48abba6c017d6a57faf2ba4b333c5449ef9596d6e08f02023c1164e5e4d0"} Jul 08 04:20:39.622019 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:39.621980 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbb8689d-8508-4493-9021-34220781027e","Type":"ContainerStarted","Data":"ee700ee98b2a5e2c4c81b33c4f236846fe5d41f59d3260068d007c38ce38cf44"} Jul 08 04:20:39.622492 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:39.622088 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbb8689d-8508-4493-9021-34220781027e","Type":"ContainerStarted","Data":"4251ccc44ddfaeb7b19912509b9660e89c0e836f504e253ef94f14f7778aacff"} Jul 08 04:20:39.622492 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:39.622134 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbb8689d-8508-4493-9021-34220781027e","Type":"ContainerStarted","Data":"d13a771920f1320870f906ca3ff457c2700962a6c9f4a2d46368303558ed5ffd"} Jul 08 04:20:39.622492 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:39.622180 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbb8689d-8508-4493-9021-34220781027e","Type":"ContainerStarted","Data":"2152f5d53646b577f71836dd343ae86c3ffff90dcc3fcd220ae76fa27a2223ca"} Jul 08 04:20:39.650626 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:39.650554 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.664334212 podStartE2EDuration="8.650534384s" totalImagesPullingTime="5.964060693s" totalInitContainerRuntime="22.139479ms" isStatefulPod=false podCreationTimestamp="2026-07-08 04:20:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:20:39.649914112 +0000 UTC m=+132.086689091" watchObservedRunningTime="2026-07-08 04:20:39.650534384 +0000 UTC m=+132.087309365" Jul 08 04:20:42.086229 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:20:42.086194 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:21:32.086264 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:21:32.086228 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:21:32.105418 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:21:32.105389 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:21:32.804794 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:21:32.804756 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 04:23:27.883732 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:27.883688 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:23:27.886061 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:27.886040 2583 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 04:23:27.886834 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:27.886813 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:23:41.866143 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:41.866105 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l9tzh"] Jul 08 04:23:41.869303 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:41.869284 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l9tzh" Jul 08 04:23:41.871773 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:41.871749 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 04:23:41.871897 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:41.871816 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 04:23:41.872971 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:41.872951 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-w29hs\"" Jul 08 04:23:41.877322 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:41.877297 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l9tzh"] Jul 08 04:23:42.013299 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:42.013254 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztm5h\" (UniqueName: \"kubernetes.io/projected/ebfe67b0-e72b-4377-a20f-7937905a92b3-kube-api-access-ztm5h\") pod \"openshift-lws-operator-fd8ccff4c-l9tzh\" (UID: \"ebfe67b0-e72b-4377-a20f-7937905a92b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l9tzh" Jul 08 04:23:42.013299 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:42.013303 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebfe67b0-e72b-4377-a20f-7937905a92b3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-l9tzh\" (UID: \"ebfe67b0-e72b-4377-a20f-7937905a92b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l9tzh" Jul 08 04:23:42.114178 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:42.114140 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztm5h\" (UniqueName: \"kubernetes.io/projected/ebfe67b0-e72b-4377-a20f-7937905a92b3-kube-api-access-ztm5h\") pod \"openshift-lws-operator-fd8ccff4c-l9tzh\" (UID: \"ebfe67b0-e72b-4377-a20f-7937905a92b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l9tzh" Jul 08 04:23:42.114178 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:42.114180 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebfe67b0-e72b-4377-a20f-7937905a92b3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-l9tzh\" (UID: \"ebfe67b0-e72b-4377-a20f-7937905a92b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l9tzh" Jul 08 04:23:42.114530 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:42.114514 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebfe67b0-e72b-4377-a20f-7937905a92b3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-l9tzh\" (UID: \"ebfe67b0-e72b-4377-a20f-7937905a92b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l9tzh" Jul 08 04:23:42.122544 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:42.122483 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztm5h\" (UniqueName: \"kubernetes.io/projected/ebfe67b0-e72b-4377-a20f-7937905a92b3-kube-api-access-ztm5h\") pod \"openshift-lws-operator-fd8ccff4c-l9tzh\" (UID: \"ebfe67b0-e72b-4377-a20f-7937905a92b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l9tzh" Jul 08 04:23:42.193538 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:42.193479 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l9tzh" Jul 08 04:23:42.310926 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:42.310841 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l9tzh"] Jul 08 04:23:42.313679 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:23:42.313650 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podebfe67b0_e72b_4377_a20f_7937905a92b3.slice/crio-d0d160a4780d041aa81298586fb887e31b267ad087c5382fcc5b26f18ad7c8c9 WatchSource:0}: Error finding container d0d160a4780d041aa81298586fb887e31b267ad087c5382fcc5b26f18ad7c8c9: Status 404 returned error can't find the container with id d0d160a4780d041aa81298586fb887e31b267ad087c5382fcc5b26f18ad7c8c9 Jul 08 04:23:42.315545 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:42.315528 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 04:23:43.210563 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:43.210521 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l9tzh" event={"ID":"ebfe67b0-e72b-4377-a20f-7937905a92b3","Type":"ContainerStarted","Data":"d0d160a4780d041aa81298586fb887e31b267ad087c5382fcc5b26f18ad7c8c9"} Jul 08 04:23:45.220034 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:45.220000 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l9tzh" event={"ID":"ebfe67b0-e72b-4377-a20f-7937905a92b3","Type":"ContainerStarted","Data":"c9f43c7f286bb3a062f7b5b82c131702eb9871f515fe264662644ab8bc512a56"} Jul 08 04:23:45.238231 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:23:45.238177 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-l9tzh" podStartSLOduration=1.430589879 podStartE2EDuration="4.238161648s" totalImagesPullingTime="2.807571769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:23:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:23:45.237036445 +0000 UTC m=+317.673811424" watchObservedRunningTime="2026-07-08 04:23:45.238161648 +0000 UTC m=+317.674936627" Jul 08 04:24:01.601520 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.601491 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v"] Jul 08 04:24:01.607465 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.607448 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:01.611724 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.611704 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 04:24:01.612124 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.612107 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 04:24:01.612247 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.612219 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 04:24:01.612309 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.612221 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 04:24:01.612370 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.612323 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qrljm\"" Jul 08 04:24:01.622483 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.622462 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v"] Jul 08 04:24:01.662366 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.662313 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ee834ca-5cac-4c2b-a1b2-2a9764e8489a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f76f945bd-csl6v\" (UID: \"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:01.662366 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.662367 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ee834ca-5cac-4c2b-a1b2-2a9764e8489a-tmp\") pod \"opendatahub-operator-controller-manager-7f76f945bd-csl6v\" (UID: \"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:01.662580 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.662399 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62f9t\" (UniqueName: \"kubernetes.io/projected/8ee834ca-5cac-4c2b-a1b2-2a9764e8489a-kube-api-access-62f9t\") pod \"opendatahub-operator-controller-manager-7f76f945bd-csl6v\" (UID: \"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:01.662580 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.662432 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8ee834ca-5cac-4c2b-a1b2-2a9764e8489a-manifests\") pod \"opendatahub-operator-controller-manager-7f76f945bd-csl6v\" (UID: \"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:01.662580 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.662474 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8ee834ca-5cac-4c2b-a1b2-2a9764e8489a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f76f945bd-csl6v\" (UID: \"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:01.762854 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.762815 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8ee834ca-5cac-4c2b-a1b2-2a9764e8489a-manifests\") pod \"opendatahub-operator-controller-manager-7f76f945bd-csl6v\" (UID: \"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:01.762854 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.762855 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8ee834ca-5cac-4c2b-a1b2-2a9764e8489a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f76f945bd-csl6v\" (UID: \"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:01.763111 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.762888 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ee834ca-5cac-4c2b-a1b2-2a9764e8489a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f76f945bd-csl6v\" (UID: \"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:01.763111 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.763012 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ee834ca-5cac-4c2b-a1b2-2a9764e8489a-tmp\") pod \"opendatahub-operator-controller-manager-7f76f945bd-csl6v\" (UID: \"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:01.763111 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.763073 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62f9t\" (UniqueName: \"kubernetes.io/projected/8ee834ca-5cac-4c2b-a1b2-2a9764e8489a-kube-api-access-62f9t\") pod \"opendatahub-operator-controller-manager-7f76f945bd-csl6v\" (UID: \"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:01.763272 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.763214 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8ee834ca-5cac-4c2b-a1b2-2a9764e8489a-manifests\") pod \"opendatahub-operator-controller-manager-7f76f945bd-csl6v\" (UID: \"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:01.765200 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.765180 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ee834ca-5cac-4c2b-a1b2-2a9764e8489a-tmp\") pod \"opendatahub-operator-controller-manager-7f76f945bd-csl6v\" (UID: \"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:01.765430 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.765410 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8ee834ca-5cac-4c2b-a1b2-2a9764e8489a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f76f945bd-csl6v\" (UID: \"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:01.765557 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.765539 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ee834ca-5cac-4c2b-a1b2-2a9764e8489a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f76f945bd-csl6v\" (UID: \"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:01.783465 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.783424 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62f9t\" (UniqueName: \"kubernetes.io/projected/8ee834ca-5cac-4c2b-a1b2-2a9764e8489a-kube-api-access-62f9t\") pod \"opendatahub-operator-controller-manager-7f76f945bd-csl6v\" (UID: \"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:01.917417 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:01.917382 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:02.050861 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:02.050827 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v"] Jul 08 04:24:02.054137 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:24:02.054111 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee834ca_5cac_4c2b_a1b2_2a9764e8489a.slice/crio-78f6d0cb9fdd8894942faccc712d6c39d81013828cf7677d6047c20958f8341c WatchSource:0}: Error finding container 78f6d0cb9fdd8894942faccc712d6c39d81013828cf7677d6047c20958f8341c: Status 404 returned error can't find the container with id 78f6d0cb9fdd8894942faccc712d6c39d81013828cf7677d6047c20958f8341c Jul 08 04:24:02.278604 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:02.278513 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" event={"ID":"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a","Type":"ContainerStarted","Data":"78f6d0cb9fdd8894942faccc712d6c39d81013828cf7677d6047c20958f8341c"} Jul 08 04:24:05.293616 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:05.293535 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" event={"ID":"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a","Type":"ContainerStarted","Data":"9e4ea78097053b4dae9895efe516e564e94e3f6df89080c407a7bc6a8f15e1a3"} Jul 08 04:24:06.297969 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:06.297935 2583 generic.go:356] "Generic (PLEG): container finished" podID="8ee834ca-5cac-4c2b-a1b2-2a9764e8489a" containerID="9e4ea78097053b4dae9895efe516e564e94e3f6df89080c407a7bc6a8f15e1a3" exitCode=0 Jul 08 04:24:06.298371 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:06.297999 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" event={"ID":"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a","Type":"ContainerDied","Data":"9e4ea78097053b4dae9895efe516e564e94e3f6df89080c407a7bc6a8f15e1a3"} Jul 08 04:24:07.303572 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:07.303532 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" event={"ID":"8ee834ca-5cac-4c2b-a1b2-2a9764e8489a","Type":"ContainerStarted","Data":"98539598c75c0a9a6685ed1cf9e787a3e7f8afe730fada00f118b87393394daa"} Jul 08 04:24:07.304033 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:07.303636 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:07.327724 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:07.327672 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" podStartSLOduration=1.956889294 podStartE2EDuration="6.327656984s" totalImagesPullingTime="3.292353277s" totalInitContainerRuntime="1.078414413s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:24:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:24:07.324093192 +0000 UTC m=+339.760868171" watchObservedRunningTime="2026-07-08 04:24:07.327656984 +0000 UTC m=+339.764431962" Jul 08 04:24:18.309729 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:18.309699 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7f76f945bd-csl6v" Jul 08 04:24:20.966654 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:20.966625 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl"] Jul 08 04:24:20.969725 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:20.969706 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" Jul 08 04:24:20.979464 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:20.979445 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 04:24:20.980595 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:20.980576 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 04:24:20.980746 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:20.980724 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 04:24:20.982287 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:20.982267 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8xv6x\"" Jul 08 04:24:21.004428 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.004379 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rznbz\" (UniqueName: \"kubernetes.io/projected/fd745bbb-d45d-4f29-98db-e9d44773d9c4-kube-api-access-rznbz\") pod \"lws-controller-manager-75c8b6477f-s6gbl\" (UID: \"fd745bbb-d45d-4f29-98db-e9d44773d9c4\") " pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" Jul 08 04:24:21.004598 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.004459 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fd745bbb-d45d-4f29-98db-e9d44773d9c4-manager-config\") pod \"lws-controller-manager-75c8b6477f-s6gbl\" (UID: \"fd745bbb-d45d-4f29-98db-e9d44773d9c4\") " pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" Jul 08 04:24:21.004598 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.004482 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd745bbb-d45d-4f29-98db-e9d44773d9c4-cert\") pod \"lws-controller-manager-75c8b6477f-s6gbl\" (UID: \"fd745bbb-d45d-4f29-98db-e9d44773d9c4\") " pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" Jul 08 04:24:21.004598 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.004504 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fd745bbb-d45d-4f29-98db-e9d44773d9c4-metrics-cert\") pod \"lws-controller-manager-75c8b6477f-s6gbl\" (UID: \"fd745bbb-d45d-4f29-98db-e9d44773d9c4\") " pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" Jul 08 04:24:21.006882 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.006849 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl"] Jul 08 04:24:21.106171 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.106139 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rznbz\" (UniqueName: \"kubernetes.io/projected/fd745bbb-d45d-4f29-98db-e9d44773d9c4-kube-api-access-rznbz\") pod \"lws-controller-manager-75c8b6477f-s6gbl\" (UID: \"fd745bbb-d45d-4f29-98db-e9d44773d9c4\") " pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" Jul 08 04:24:21.106372 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.106248 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fd745bbb-d45d-4f29-98db-e9d44773d9c4-manager-config\") pod \"lws-controller-manager-75c8b6477f-s6gbl\" (UID: \"fd745bbb-d45d-4f29-98db-e9d44773d9c4\") " pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" Jul 08 04:24:21.106372 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.106304 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd745bbb-d45d-4f29-98db-e9d44773d9c4-cert\") pod \"lws-controller-manager-75c8b6477f-s6gbl\" (UID: \"fd745bbb-d45d-4f29-98db-e9d44773d9c4\") " pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" Jul 08 04:24:21.106372 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.106333 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fd745bbb-d45d-4f29-98db-e9d44773d9c4-metrics-cert\") pod \"lws-controller-manager-75c8b6477f-s6gbl\" (UID: \"fd745bbb-d45d-4f29-98db-e9d44773d9c4\") " pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" Jul 08 04:24:21.107223 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.107196 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fd745bbb-d45d-4f29-98db-e9d44773d9c4-manager-config\") pod \"lws-controller-manager-75c8b6477f-s6gbl\" (UID: \"fd745bbb-d45d-4f29-98db-e9d44773d9c4\") " pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" Jul 08 04:24:21.108904 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.108880 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fd745bbb-d45d-4f29-98db-e9d44773d9c4-metrics-cert\") pod \"lws-controller-manager-75c8b6477f-s6gbl\" (UID: \"fd745bbb-d45d-4f29-98db-e9d44773d9c4\") " pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" Jul 08 04:24:21.109189 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.109172 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd745bbb-d45d-4f29-98db-e9d44773d9c4-cert\") pod \"lws-controller-manager-75c8b6477f-s6gbl\" (UID: \"fd745bbb-d45d-4f29-98db-e9d44773d9c4\") " pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" Jul 08 04:24:21.138102 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.138073 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rznbz\" (UniqueName: \"kubernetes.io/projected/fd745bbb-d45d-4f29-98db-e9d44773d9c4-kube-api-access-rznbz\") pod \"lws-controller-manager-75c8b6477f-s6gbl\" (UID: \"fd745bbb-d45d-4f29-98db-e9d44773d9c4\") " pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" Jul 08 04:24:21.278522 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.278438 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" Jul 08 04:24:21.416859 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.416824 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl"] Jul 08 04:24:21.419528 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:24:21.419501 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd745bbb_d45d_4f29_98db_e9d44773d9c4.slice/crio-dab57efb022aeffe80d9d3e9c1d65f9f43acc3e4f38a83abfe61db032ba3f8f9 WatchSource:0}: Error finding container dab57efb022aeffe80d9d3e9c1d65f9f43acc3e4f38a83abfe61db032ba3f8f9: Status 404 returned error can't find the container with id dab57efb022aeffe80d9d3e9c1d65f9f43acc3e4f38a83abfe61db032ba3f8f9 Jul 08 04:24:21.718003 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.717966 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c8bhm"] Jul 08 04:24:21.722421 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.722401 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-c8bhm" Jul 08 04:24:21.724902 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.724879 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 08 04:24:21.724998 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.724986 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-hs6jw\"" Jul 08 04:24:21.729890 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.729867 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c8bhm"] Jul 08 04:24:21.811106 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.811079 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/313b79f2-b436-4920-92c7-04fdddf2e58c-cert\") pod \"odh-model-controller-7998c67dc4-c8bhm\" (UID: \"313b79f2-b436-4920-92c7-04fdddf2e58c\") " pod="opendatahub/odh-model-controller-7998c67dc4-c8bhm" Jul 08 04:24:21.811263 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.811138 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-646fb\" (UniqueName: \"kubernetes.io/projected/313b79f2-b436-4920-92c7-04fdddf2e58c-kube-api-access-646fb\") pod \"odh-model-controller-7998c67dc4-c8bhm\" (UID: \"313b79f2-b436-4920-92c7-04fdddf2e58c\") " pod="opendatahub/odh-model-controller-7998c67dc4-c8bhm" Jul 08 04:24:21.912285 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.912256 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-646fb\" (UniqueName: \"kubernetes.io/projected/313b79f2-b436-4920-92c7-04fdddf2e58c-kube-api-access-646fb\") pod \"odh-model-controller-7998c67dc4-c8bhm\" (UID: \"313b79f2-b436-4920-92c7-04fdddf2e58c\") " pod="opendatahub/odh-model-controller-7998c67dc4-c8bhm" Jul 08 04:24:21.912473 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.912306 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/313b79f2-b436-4920-92c7-04fdddf2e58c-cert\") pod \"odh-model-controller-7998c67dc4-c8bhm\" (UID: \"313b79f2-b436-4920-92c7-04fdddf2e58c\") " pod="opendatahub/odh-model-controller-7998c67dc4-c8bhm" Jul 08 04:24:21.914572 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.914556 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/313b79f2-b436-4920-92c7-04fdddf2e58c-cert\") pod \"odh-model-controller-7998c67dc4-c8bhm\" (UID: \"313b79f2-b436-4920-92c7-04fdddf2e58c\") " pod="opendatahub/odh-model-controller-7998c67dc4-c8bhm" Jul 08 04:24:21.925589 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:21.925569 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-646fb\" (UniqueName: \"kubernetes.io/projected/313b79f2-b436-4920-92c7-04fdddf2e58c-kube-api-access-646fb\") pod \"odh-model-controller-7998c67dc4-c8bhm\" (UID: \"313b79f2-b436-4920-92c7-04fdddf2e58c\") " pod="opendatahub/odh-model-controller-7998c67dc4-c8bhm" Jul 08 04:24:22.033312 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:22.033236 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-c8bhm" Jul 08 04:24:22.174691 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:22.174657 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-c8bhm"] Jul 08 04:24:22.178465 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:24:22.178437 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod313b79f2_b436_4920_92c7_04fdddf2e58c.slice/crio-7dd8f54b77abf02be65d480b72c3595196872291c3c16c557e4bcf78f55a8471 WatchSource:0}: Error finding container 7dd8f54b77abf02be65d480b72c3595196872291c3c16c557e4bcf78f55a8471: Status 404 returned error can't find the container with id 7dd8f54b77abf02be65d480b72c3595196872291c3c16c557e4bcf78f55a8471 Jul 08 04:24:22.341423 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:22.341337 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-h4qqn"] Jul 08 04:24:22.346100 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:22.346080 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-h4qqn" Jul 08 04:24:22.348488 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:22.348462 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 08 04:24:22.356010 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:22.355989 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-h4qqn"] Jul 08 04:24:22.356949 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:22.356924 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-c8bhm" event={"ID":"313b79f2-b436-4920-92c7-04fdddf2e58c","Type":"ContainerStarted","Data":"7dd8f54b77abf02be65d480b72c3595196872291c3c16c557e4bcf78f55a8471"} Jul 08 04:24:22.358192 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:22.358172 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" event={"ID":"fd745bbb-d45d-4f29-98db-e9d44773d9c4","Type":"ContainerStarted","Data":"dab57efb022aeffe80d9d3e9c1d65f9f43acc3e4f38a83abfe61db032ba3f8f9"} Jul 08 04:24:22.416152 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:22.416121 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d62104b-a8cf-46d9-bafa-4f0580fe4575-tls-certs\") pod \"model-serving-api-54d89bd5fc-h4qqn\" (UID: \"0d62104b-a8cf-46d9-bafa-4f0580fe4575\") " pod="opendatahub/model-serving-api-54d89bd5fc-h4qqn" Jul 08 04:24:22.416276 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:22.416161 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wt8t\" (UniqueName: \"kubernetes.io/projected/0d62104b-a8cf-46d9-bafa-4f0580fe4575-kube-api-access-6wt8t\") pod \"model-serving-api-54d89bd5fc-h4qqn\" (UID: \"0d62104b-a8cf-46d9-bafa-4f0580fe4575\") " pod="opendatahub/model-serving-api-54d89bd5fc-h4qqn" Jul 08 04:24:22.517010 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:22.516977 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d62104b-a8cf-46d9-bafa-4f0580fe4575-tls-certs\") pod \"model-serving-api-54d89bd5fc-h4qqn\" (UID: \"0d62104b-a8cf-46d9-bafa-4f0580fe4575\") " pod="opendatahub/model-serving-api-54d89bd5fc-h4qqn" Jul 08 04:24:22.517010 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:22.517013 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wt8t\" (UniqueName: \"kubernetes.io/projected/0d62104b-a8cf-46d9-bafa-4f0580fe4575-kube-api-access-6wt8t\") pod \"model-serving-api-54d89bd5fc-h4qqn\" (UID: \"0d62104b-a8cf-46d9-bafa-4f0580fe4575\") " pod="opendatahub/model-serving-api-54d89bd5fc-h4qqn" Jul 08 04:24:22.519314 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:22.519289 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d62104b-a8cf-46d9-bafa-4f0580fe4575-tls-certs\") pod \"model-serving-api-54d89bd5fc-h4qqn\" (UID: \"0d62104b-a8cf-46d9-bafa-4f0580fe4575\") " pod="opendatahub/model-serving-api-54d89bd5fc-h4qqn" Jul 08 04:24:22.525442 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:22.525420 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wt8t\" (UniqueName: \"kubernetes.io/projected/0d62104b-a8cf-46d9-bafa-4f0580fe4575-kube-api-access-6wt8t\") pod \"model-serving-api-54d89bd5fc-h4qqn\" (UID: \"0d62104b-a8cf-46d9-bafa-4f0580fe4575\") " pod="opendatahub/model-serving-api-54d89bd5fc-h4qqn" Jul 08 04:24:22.657837 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:22.657768 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-h4qqn" Jul 08 04:24:22.800511 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:22.800481 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-h4qqn"] Jul 08 04:24:22.803974 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:24:22.803945 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d62104b_a8cf_46d9_bafa_4f0580fe4575.slice/crio-935472fe5e499c6b8c7c3aea7a118b90821ca742f59d840e0544440935c233a9 WatchSource:0}: Error finding container 935472fe5e499c6b8c7c3aea7a118b90821ca742f59d840e0544440935c233a9: Status 404 returned error can't find the container with id 935472fe5e499c6b8c7c3aea7a118b90821ca742f59d840e0544440935c233a9 Jul 08 04:24:23.364427 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:23.364330 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-h4qqn" event={"ID":"0d62104b-a8cf-46d9-bafa-4f0580fe4575","Type":"ContainerStarted","Data":"935472fe5e499c6b8c7c3aea7a118b90821ca742f59d840e0544440935c233a9"} Jul 08 04:24:26.377421 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:26.377316 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-c8bhm" event={"ID":"313b79f2-b436-4920-92c7-04fdddf2e58c","Type":"ContainerStarted","Data":"380a0ffe94753d0bf61e2de8450f5fb85c16485d6593eeb618fef1c38823c89b"} Jul 08 04:24:26.377421 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:26.377406 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-c8bhm" Jul 08 04:24:26.379003 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:26.378969 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-h4qqn" event={"ID":"0d62104b-a8cf-46d9-bafa-4f0580fe4575","Type":"ContainerStarted","Data":"71072861200a8f3c8fceae58ab096f113b808269863b96f8ed5ae94d15b7cb85"} Jul 08 04:24:26.379130 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:26.379028 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-h4qqn" Jul 08 04:24:26.381263 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:26.381235 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" event={"ID":"fd745bbb-d45d-4f29-98db-e9d44773d9c4","Type":"ContainerStarted","Data":"89d10f19fbaee305b41fd67a877e1bc0b35db85161af9e277c5565bb77d4f17b"} Jul 08 04:24:26.381394 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:26.381291 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" Jul 08 04:24:26.394542 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:26.394494 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-c8bhm" podStartSLOduration=1.4961918810000001 podStartE2EDuration="5.394479121s" totalImagesPullingTime="3.89828724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:24:26.392310861 +0000 UTC m=+358.829085839" watchObservedRunningTime="2026-07-08 04:24:26.394479121 +0000 UTC m=+358.831254100" Jul 08 04:24:26.409059 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:26.409002 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-h4qqn" podStartSLOduration=1.137044561 podStartE2EDuration="4.408989301s" totalImagesPullingTime="3.27194474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:24:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:24:26.407842752 +0000 UTC m=+358.844617745" watchObservedRunningTime="2026-07-08 04:24:26.408989301 +0000 UTC m=+358.845764280" Jul 08 04:24:26.423333 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:26.423283 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" podStartSLOduration=1.781210045 podStartE2EDuration="6.42326559s" totalImagesPullingTime="4.642055545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:24:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:24:26.422969317 +0000 UTC m=+358.859744318" watchObservedRunningTime="2026-07-08 04:24:26.42326559 +0000 UTC m=+358.860040574" Jul 08 04:24:27.070900 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.070863 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-668854d458-4ctbp"] Jul 08 04:24:27.084470 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.084422 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-668854d458-4ctbp"] Jul 08 04:24:27.084627 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.084569 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-668854d458-4ctbp" Jul 08 04:24:27.087556 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.087511 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 04:24:27.087723 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.087640 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 04:24:27.087792 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.087730 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 04:24:27.087847 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.087825 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 04:24:27.087907 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.087854 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-rbnwh\"" Jul 08 04:24:27.161620 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.161583 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mfbt\" (UniqueName: \"kubernetes.io/projected/d847cbd1-645f-4266-8ad4-05ae2ac443e1-kube-api-access-8mfbt\") pod \"kube-auth-proxy-668854d458-4ctbp\" (UID: \"d847cbd1-645f-4266-8ad4-05ae2ac443e1\") " pod="openshift-ingress/kube-auth-proxy-668854d458-4ctbp" Jul 08 04:24:27.161834 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.161657 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d847cbd1-645f-4266-8ad4-05ae2ac443e1-tmp\") pod \"kube-auth-proxy-668854d458-4ctbp\" (UID: \"d847cbd1-645f-4266-8ad4-05ae2ac443e1\") " pod="openshift-ingress/kube-auth-proxy-668854d458-4ctbp" Jul 08 04:24:27.161834 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.161755 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d847cbd1-645f-4266-8ad4-05ae2ac443e1-tls-certs\") pod \"kube-auth-proxy-668854d458-4ctbp\" (UID: \"d847cbd1-645f-4266-8ad4-05ae2ac443e1\") " pod="openshift-ingress/kube-auth-proxy-668854d458-4ctbp" Jul 08 04:24:27.263025 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.262959 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mfbt\" (UniqueName: \"kubernetes.io/projected/d847cbd1-645f-4266-8ad4-05ae2ac443e1-kube-api-access-8mfbt\") pod \"kube-auth-proxy-668854d458-4ctbp\" (UID: \"d847cbd1-645f-4266-8ad4-05ae2ac443e1\") " pod="openshift-ingress/kube-auth-proxy-668854d458-4ctbp" Jul 08 04:24:27.263025 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.263034 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d847cbd1-645f-4266-8ad4-05ae2ac443e1-tmp\") pod \"kube-auth-proxy-668854d458-4ctbp\" (UID: \"d847cbd1-645f-4266-8ad4-05ae2ac443e1\") " pod="openshift-ingress/kube-auth-proxy-668854d458-4ctbp" Jul 08 04:24:27.263386 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.263078 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d847cbd1-645f-4266-8ad4-05ae2ac443e1-tls-certs\") pod \"kube-auth-proxy-668854d458-4ctbp\" (UID: \"d847cbd1-645f-4266-8ad4-05ae2ac443e1\") " pod="openshift-ingress/kube-auth-proxy-668854d458-4ctbp" Jul 08 04:24:27.265485 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.265457 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d847cbd1-645f-4266-8ad4-05ae2ac443e1-tmp\") pod \"kube-auth-proxy-668854d458-4ctbp\" (UID: \"d847cbd1-645f-4266-8ad4-05ae2ac443e1\") " pod="openshift-ingress/kube-auth-proxy-668854d458-4ctbp" Jul 08 04:24:27.265694 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.265673 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d847cbd1-645f-4266-8ad4-05ae2ac443e1-tls-certs\") pod \"kube-auth-proxy-668854d458-4ctbp\" (UID: \"d847cbd1-645f-4266-8ad4-05ae2ac443e1\") " pod="openshift-ingress/kube-auth-proxy-668854d458-4ctbp" Jul 08 04:24:27.272587 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.272556 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mfbt\" (UniqueName: \"kubernetes.io/projected/d847cbd1-645f-4266-8ad4-05ae2ac443e1-kube-api-access-8mfbt\") pod \"kube-auth-proxy-668854d458-4ctbp\" (UID: \"d847cbd1-645f-4266-8ad4-05ae2ac443e1\") " pod="openshift-ingress/kube-auth-proxy-668854d458-4ctbp" Jul 08 04:24:27.395988 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.395903 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-668854d458-4ctbp" Jul 08 04:24:27.530636 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:27.530614 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-668854d458-4ctbp"] Jul 08 04:24:27.533228 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:24:27.533190 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd847cbd1_645f_4266_8ad4_05ae2ac443e1.slice/crio-de8a4f719388ca41df0277c3706a47a48e5a182ada2a8e84f0b1d30c582d79f0 WatchSource:0}: Error finding container de8a4f719388ca41df0277c3706a47a48e5a182ada2a8e84f0b1d30c582d79f0: Status 404 returned error can't find the container with id de8a4f719388ca41df0277c3706a47a48e5a182ada2a8e84f0b1d30c582d79f0 Jul 08 04:24:28.390409 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:28.390367 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-668854d458-4ctbp" event={"ID":"d847cbd1-645f-4266-8ad4-05ae2ac443e1","Type":"ContainerStarted","Data":"de8a4f719388ca41df0277c3706a47a48e5a182ada2a8e84f0b1d30c582d79f0"} Jul 08 04:24:30.376526 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:30.376503 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 04:24:31.403963 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:31.403929 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-668854d458-4ctbp" event={"ID":"d847cbd1-645f-4266-8ad4-05ae2ac443e1","Type":"ContainerStarted","Data":"4aacf8272fda42182893a310c5d1e7c5c9cfec491c727a62fad9023ec28c751e"} Jul 08 04:24:31.423375 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:31.423307 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-668854d458-4ctbp" podStartSLOduration=1.584691157 podStartE2EDuration="4.42328691s" totalImagesPullingTime="2.838595753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:24:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:24:31.421730008 +0000 UTC m=+363.858505001" watchObservedRunningTime="2026-07-08 04:24:31.42328691 +0000 UTC m=+363.860061890" Jul 08 04:24:32.447379 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:32.447336 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq"] Jul 08 04:24:32.450730 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:32.450712 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq" Jul 08 04:24:32.453524 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:32.453499 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-dvmxq\"" Jul 08 04:24:32.453524 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:32.453510 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 08 04:24:32.462404 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:32.462338 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq"] Jul 08 04:24:32.506366 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:32.506320 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a0cf4c9-10e4-4904-b1aa-3dba4d42e707-cert\") pod \"kserve-controller-manager-d6f4b65b8-7h8bq\" (UID: \"5a0cf4c9-10e4-4904-b1aa-3dba4d42e707\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq" Jul 08 04:24:32.506553 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:32.506381 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8hff\" (UniqueName: \"kubernetes.io/projected/5a0cf4c9-10e4-4904-b1aa-3dba4d42e707-kube-api-access-w8hff\") pod \"kserve-controller-manager-d6f4b65b8-7h8bq\" (UID: \"5a0cf4c9-10e4-4904-b1aa-3dba4d42e707\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq" Jul 08 04:24:32.607685 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:32.607653 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a0cf4c9-10e4-4904-b1aa-3dba4d42e707-cert\") pod \"kserve-controller-manager-d6f4b65b8-7h8bq\" (UID: \"5a0cf4c9-10e4-4904-b1aa-3dba4d42e707\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq" Jul 08 04:24:32.607896 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:32.607691 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8hff\" (UniqueName: \"kubernetes.io/projected/5a0cf4c9-10e4-4904-b1aa-3dba4d42e707-kube-api-access-w8hff\") pod \"kserve-controller-manager-d6f4b65b8-7h8bq\" (UID: \"5a0cf4c9-10e4-4904-b1aa-3dba4d42e707\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq" Jul 08 04:24:32.607896 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:24:32.607796 2583 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 08 04:24:32.607896 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:24:32.607856 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a0cf4c9-10e4-4904-b1aa-3dba4d42e707-cert podName:5a0cf4c9-10e4-4904-b1aa-3dba4d42e707 nodeName:}" failed. No retries permitted until 2026-07-08 04:24:33.107841317 +0000 UTC m=+365.544616273 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5a0cf4c9-10e4-4904-b1aa-3dba4d42e707-cert") pod "kserve-controller-manager-d6f4b65b8-7h8bq" (UID: "5a0cf4c9-10e4-4904-b1aa-3dba4d42e707") : secret "kserve-webhook-server-cert" not found Jul 08 04:24:32.616792 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:32.616768 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8hff\" (UniqueName: \"kubernetes.io/projected/5a0cf4c9-10e4-4904-b1aa-3dba4d42e707-kube-api-access-w8hff\") pod \"kserve-controller-manager-d6f4b65b8-7h8bq\" (UID: \"5a0cf4c9-10e4-4904-b1aa-3dba4d42e707\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq" Jul 08 04:24:33.112031 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:33.111998 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a0cf4c9-10e4-4904-b1aa-3dba4d42e707-cert\") pod \"kserve-controller-manager-d6f4b65b8-7h8bq\" (UID: \"5a0cf4c9-10e4-4904-b1aa-3dba4d42e707\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq" Jul 08 04:24:33.114686 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:33.114662 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a0cf4c9-10e4-4904-b1aa-3dba4d42e707-cert\") pod \"kserve-controller-manager-d6f4b65b8-7h8bq\" (UID: \"5a0cf4c9-10e4-4904-b1aa-3dba4d42e707\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq" Jul 08 04:24:33.361131 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:33.361098 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq" Jul 08 04:24:33.513971 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:33.513932 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq"] Jul 08 04:24:33.516082 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:24:33.516055 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a0cf4c9_10e4_4904_b1aa_3dba4d42e707.slice/crio-a7d8fb0447c5de8a1186c27b9c85526c7f3d09e9c45041eecc8c2bcb28a6f215 WatchSource:0}: Error finding container a7d8fb0447c5de8a1186c27b9c85526c7f3d09e9c45041eecc8c2bcb28a6f215: Status 404 returned error can't find the container with id a7d8fb0447c5de8a1186c27b9c85526c7f3d09e9c45041eecc8c2bcb28a6f215 Jul 08 04:24:34.419018 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:34.418986 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq" event={"ID":"5a0cf4c9-10e4-4904-b1aa-3dba4d42e707","Type":"ContainerStarted","Data":"a7d8fb0447c5de8a1186c27b9c85526c7f3d09e9c45041eecc8c2bcb28a6f215"} Jul 08 04:24:36.428779 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:36.428734 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq" event={"ID":"5a0cf4c9-10e4-4904-b1aa-3dba4d42e707","Type":"ContainerStarted","Data":"12d04a833d7daed02f4ce76081befb22b89e3d1488aacffe5b62cf9652e22499"} Jul 08 04:24:36.429217 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:36.428818 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq" Jul 08 04:24:36.450418 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:36.450372 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq" podStartSLOduration=1.851443782 podStartE2EDuration="4.450333618s" totalImagesPullingTime="2.598889836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:24:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:24:36.449420883 +0000 UTC m=+368.886195863" watchObservedRunningTime="2026-07-08 04:24:36.450333618 +0000 UTC m=+368.887108597" Jul 08 04:24:37.387483 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:37.387455 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-c8bhm" Jul 08 04:24:37.387748 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:37.387728 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-75c8b6477f-s6gbl" Jul 08 04:24:37.389875 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:24:37.389854 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-h4qqn" Jul 08 04:25:07.438269 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:25:07.438238 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7h8bq" Jul 08 04:26:15.266444 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:15.266356 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm"] Jul 08 04:26:15.269255 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:15.269234 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm" Jul 08 04:26:15.272053 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:15.272031 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 08 04:26:15.272053 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:15.272050 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 08 04:26:15.272186 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:15.272051 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 04:26:15.272186 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:15.272126 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 04:26:15.273206 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:15.273189 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9wxhx\"" Jul 08 04:26:15.284169 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:15.284149 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm"] Jul 08 04:26:15.404073 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:15.404035 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/747a0f10-9ba7-4402-b601-dbcf0d2ecf66-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-prbsm\" (UID: \"747a0f10-9ba7-4402-b601-dbcf0d2ecf66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm" Jul 08 04:26:15.404239 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:15.404091 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzdzv\" (UniqueName: \"kubernetes.io/projected/747a0f10-9ba7-4402-b601-dbcf0d2ecf66-kube-api-access-mzdzv\") pod \"kuadrant-console-plugin-66b5774985-prbsm\" (UID: \"747a0f10-9ba7-4402-b601-dbcf0d2ecf66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm" Jul 08 04:26:15.404239 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:15.404118 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/747a0f10-9ba7-4402-b601-dbcf0d2ecf66-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-prbsm\" (UID: \"747a0f10-9ba7-4402-b601-dbcf0d2ecf66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm" Jul 08 04:26:15.505122 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:15.505085 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/747a0f10-9ba7-4402-b601-dbcf0d2ecf66-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-prbsm\" (UID: \"747a0f10-9ba7-4402-b601-dbcf0d2ecf66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm" Jul 08 04:26:15.505281 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:15.505145 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/747a0f10-9ba7-4402-b601-dbcf0d2ecf66-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-prbsm\" (UID: \"747a0f10-9ba7-4402-b601-dbcf0d2ecf66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm" Jul 08 04:26:15.505281 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:15.505187 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzdzv\" (UniqueName: \"kubernetes.io/projected/747a0f10-9ba7-4402-b601-dbcf0d2ecf66-kube-api-access-mzdzv\") pod \"kuadrant-console-plugin-66b5774985-prbsm\" (UID: \"747a0f10-9ba7-4402-b601-dbcf0d2ecf66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm" Jul 08 04:26:15.505281 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:26:15.505228 2583 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 08 04:26:15.505419 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:26:15.505308 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/747a0f10-9ba7-4402-b601-dbcf0d2ecf66-plugin-serving-cert podName:747a0f10-9ba7-4402-b601-dbcf0d2ecf66 nodeName:}" failed. No retries permitted until 2026-07-08 04:26:16.005290826 +0000 UTC m=+468.442065788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/747a0f10-9ba7-4402-b601-dbcf0d2ecf66-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-prbsm" (UID: "747a0f10-9ba7-4402-b601-dbcf0d2ecf66") : secret "plugin-serving-cert" not found Jul 08 04:26:15.505800 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:15.505782 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/747a0f10-9ba7-4402-b601-dbcf0d2ecf66-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-prbsm\" (UID: \"747a0f10-9ba7-4402-b601-dbcf0d2ecf66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm" Jul 08 04:26:15.514374 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:15.514336 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzdzv\" (UniqueName: \"kubernetes.io/projected/747a0f10-9ba7-4402-b601-dbcf0d2ecf66-kube-api-access-mzdzv\") pod \"kuadrant-console-plugin-66b5774985-prbsm\" (UID: \"747a0f10-9ba7-4402-b601-dbcf0d2ecf66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm" Jul 08 04:26:16.010607 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:16.010566 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/747a0f10-9ba7-4402-b601-dbcf0d2ecf66-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-prbsm\" (UID: \"747a0f10-9ba7-4402-b601-dbcf0d2ecf66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm" Jul 08 04:26:16.012991 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:16.012971 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/747a0f10-9ba7-4402-b601-dbcf0d2ecf66-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-prbsm\" (UID: \"747a0f10-9ba7-4402-b601-dbcf0d2ecf66\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm" Jul 08 04:26:16.178222 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:16.178190 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm" Jul 08 04:26:16.318868 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:16.318836 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm"] Jul 08 04:26:16.322048 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:26:16.322012 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod747a0f10_9ba7_4402_b601_dbcf0d2ecf66.slice/crio-15e46d3180d877ea48f8912463b61c0de79000eb0025ab69370a26ddc4d6a425 WatchSource:0}: Error finding container 15e46d3180d877ea48f8912463b61c0de79000eb0025ab69370a26ddc4d6a425: Status 404 returned error can't find the container with id 15e46d3180d877ea48f8912463b61c0de79000eb0025ab69370a26ddc4d6a425 Jul 08 04:26:16.814763 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:16.814732 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm" event={"ID":"747a0f10-9ba7-4402-b601-dbcf0d2ecf66","Type":"ContainerStarted","Data":"15e46d3180d877ea48f8912463b61c0de79000eb0025ab69370a26ddc4d6a425"} Jul 08 04:26:26.420480 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:26.416716 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs"] Jul 08 04:26:26.426553 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:26.426527 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" Jul 08 04:26:26.429872 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:26.429763 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-p6hqn\"" Jul 08 04:26:26.431816 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:26.431756 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs"] Jul 08 04:26:26.507871 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:26.507802 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxzr8\" (UniqueName: \"kubernetes.io/projected/29c57b64-e155-4259-9599-0d6815e4dc8d-kube-api-access-fxzr8\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rm6rs\" (UID: \"29c57b64-e155-4259-9599-0d6815e4dc8d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" Jul 08 04:26:26.508030 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:26.507879 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29c57b64-e155-4259-9599-0d6815e4dc8d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rm6rs\" (UID: \"29c57b64-e155-4259-9599-0d6815e4dc8d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" Jul 08 04:26:26.609381 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:26.609332 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29c57b64-e155-4259-9599-0d6815e4dc8d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rm6rs\" (UID: \"29c57b64-e155-4259-9599-0d6815e4dc8d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" Jul 08 04:26:26.609574 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:26.609432 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxzr8\" (UniqueName: \"kubernetes.io/projected/29c57b64-e155-4259-9599-0d6815e4dc8d-kube-api-access-fxzr8\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rm6rs\" (UID: \"29c57b64-e155-4259-9599-0d6815e4dc8d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" Jul 08 04:26:26.609758 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:26.609732 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29c57b64-e155-4259-9599-0d6815e4dc8d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rm6rs\" (UID: \"29c57b64-e155-4259-9599-0d6815e4dc8d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" Jul 08 04:26:26.630123 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:26.630095 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxzr8\" (UniqueName: \"kubernetes.io/projected/29c57b64-e155-4259-9599-0d6815e4dc8d-kube-api-access-fxzr8\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-rm6rs\" (UID: \"29c57b64-e155-4259-9599-0d6815e4dc8d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" Jul 08 04:26:26.741651 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:26.741610 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" Jul 08 04:26:40.334912 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:40.334877 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs"] Jul 08 04:26:40.337937 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:26:40.337908 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29c57b64_e155_4259_9599_0d6815e4dc8d.slice/crio-fd37e9bd1f9330bd0fdafaa5a33285a6b1f38a4dbe5e8e2d622464b987a462f2 WatchSource:0}: Error finding container fd37e9bd1f9330bd0fdafaa5a33285a6b1f38a4dbe5e8e2d622464b987a462f2: Status 404 returned error can't find the container with id fd37e9bd1f9330bd0fdafaa5a33285a6b1f38a4dbe5e8e2d622464b987a462f2 Jul 08 04:26:40.928699 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:40.928616 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" event={"ID":"29c57b64-e155-4259-9599-0d6815e4dc8d","Type":"ContainerStarted","Data":"fd37e9bd1f9330bd0fdafaa5a33285a6b1f38a4dbe5e8e2d622464b987a462f2"} Jul 08 04:26:40.930003 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:40.929978 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm" event={"ID":"747a0f10-9ba7-4402-b601-dbcf0d2ecf66","Type":"ContainerStarted","Data":"1a0808b34f3d0b1011222b3440c5b47157916d629ba27115bb1d2a52a8ae6bcf"} Jul 08 04:26:46.958795 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:46.958763 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" event={"ID":"29c57b64-e155-4259-9599-0d6815e4dc8d","Type":"ContainerStarted","Data":"8d5e9ae373b5c0a134a59cf7dfb465ae4559f542aab1f0a2c6cee82477375a9a"} Jul 08 04:26:46.959193 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:46.958834 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" Jul 08 04:26:46.977650 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:46.977607 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-prbsm" podStartSLOduration=7.6514643719999995 podStartE2EDuration="31.977592348s" totalImagesPullingTime="24.326127976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:26:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:26:40.945661946 +0000 UTC m=+493.382436925" watchObservedRunningTime="2026-07-08 04:26:46.977592348 +0000 UTC m=+499.414367348" Jul 08 04:26:46.982922 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:46.979464 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" podStartSLOduration=15.216019621 podStartE2EDuration="20.979452222s" totalImagesPullingTime="5.763432601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:26:46.975992026 +0000 UTC m=+499.412767004" watchObservedRunningTime="2026-07-08 04:26:46.979452222 +0000 UTC m=+499.416227201" Jul 08 04:26:57.965828 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:26:57.965796 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" Jul 08 04:27:10.728118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:10.728082 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs"] Jul 08 04:27:10.728569 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:10.728281 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" podUID="29c57b64-e155-4259-9599-0d6815e4dc8d" containerName="manager" containerID="cri-o://8d5e9ae373b5c0a134a59cf7dfb465ae4559f542aab1f0a2c6cee82477375a9a" gracePeriod=10 Jul 08 04:27:10.977331 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:10.977307 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" Jul 08 04:27:10.981801 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:10.981759 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxzr8\" (UniqueName: \"kubernetes.io/projected/29c57b64-e155-4259-9599-0d6815e4dc8d-kube-api-access-fxzr8\") pod \"29c57b64-e155-4259-9599-0d6815e4dc8d\" (UID: \"29c57b64-e155-4259-9599-0d6815e4dc8d\") " Jul 08 04:27:10.981876 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:10.981818 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29c57b64-e155-4259-9599-0d6815e4dc8d-extensions-socket-volume\") pod \"29c57b64-e155-4259-9599-0d6815e4dc8d\" (UID: \"29c57b64-e155-4259-9599-0d6815e4dc8d\") " Jul 08 04:27:10.982250 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:10.982228 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c57b64-e155-4259-9599-0d6815e4dc8d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "29c57b64-e155-4259-9599-0d6815e4dc8d" (UID: "29c57b64-e155-4259-9599-0d6815e4dc8d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:27:10.983834 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:10.983810 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29c57b64-e155-4259-9599-0d6815e4dc8d-kube-api-access-fxzr8" (OuterVolumeSpecName: "kube-api-access-fxzr8") pod "29c57b64-e155-4259-9599-0d6815e4dc8d" (UID: "29c57b64-e155-4259-9599-0d6815e4dc8d"). InnerVolumeSpecName "kube-api-access-fxzr8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:27:11.064000 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:11.063964 2583 generic.go:356] "Generic (PLEG): container finished" podID="29c57b64-e155-4259-9599-0d6815e4dc8d" containerID="8d5e9ae373b5c0a134a59cf7dfb465ae4559f542aab1f0a2c6cee82477375a9a" exitCode=0 Jul 08 04:27:11.064156 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:11.064016 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" Jul 08 04:27:11.064156 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:11.064037 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" event={"ID":"29c57b64-e155-4259-9599-0d6815e4dc8d","Type":"ContainerDied","Data":"8d5e9ae373b5c0a134a59cf7dfb465ae4559f542aab1f0a2c6cee82477375a9a"} Jul 08 04:27:11.064156 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:11.064116 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs" event={"ID":"29c57b64-e155-4259-9599-0d6815e4dc8d","Type":"ContainerDied","Data":"fd37e9bd1f9330bd0fdafaa5a33285a6b1f38a4dbe5e8e2d622464b987a462f2"} Jul 08 04:27:11.064156 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:11.064145 2583 scope.go:117] "RemoveContainer" containerID="8d5e9ae373b5c0a134a59cf7dfb465ae4559f542aab1f0a2c6cee82477375a9a" Jul 08 04:27:11.072925 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:11.072905 2583 scope.go:117] "RemoveContainer" containerID="8d5e9ae373b5c0a134a59cf7dfb465ae4559f542aab1f0a2c6cee82477375a9a" Jul 08 04:27:11.073188 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:27:11.073169 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d5e9ae373b5c0a134a59cf7dfb465ae4559f542aab1f0a2c6cee82477375a9a\": container with ID starting with 8d5e9ae373b5c0a134a59cf7dfb465ae4559f542aab1f0a2c6cee82477375a9a not found: ID does not exist" containerID="8d5e9ae373b5c0a134a59cf7dfb465ae4559f542aab1f0a2c6cee82477375a9a" Jul 08 04:27:11.073238 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:11.073197 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d5e9ae373b5c0a134a59cf7dfb465ae4559f542aab1f0a2c6cee82477375a9a"} err="failed to get container status \"8d5e9ae373b5c0a134a59cf7dfb465ae4559f542aab1f0a2c6cee82477375a9a\": rpc error: code = NotFound desc = could not find container \"8d5e9ae373b5c0a134a59cf7dfb465ae4559f542aab1f0a2c6cee82477375a9a\": container with ID starting with 8d5e9ae373b5c0a134a59cf7dfb465ae4559f542aab1f0a2c6cee82477375a9a not found: ID does not exist" Jul 08 04:27:11.082309 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:11.082287 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29c57b64-e155-4259-9599-0d6815e4dc8d-extensions-socket-volume\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:27:11.082309 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:11.082308 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxzr8\" (UniqueName: \"kubernetes.io/projected/29c57b64-e155-4259-9599-0d6815e4dc8d-kube-api-access-fxzr8\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:27:11.085946 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:11.085926 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs"] Jul 08 04:27:11.092027 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:11.092009 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-rm6rs"] Jul 08 04:27:12.072422 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:12.072385 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29c57b64-e155-4259-9599-0d6815e4dc8d" path="/var/lib/kubelet/pods/29c57b64-e155-4259-9599-0d6815e4dc8d/volumes" Jul 08 04:27:20.271919 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.271840 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:27:20.272367 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.272119 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29c57b64-e155-4259-9599-0d6815e4dc8d" containerName="manager" Jul 08 04:27:20.272367 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.272131 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="29c57b64-e155-4259-9599-0d6815e4dc8d" containerName="manager" Jul 08 04:27:20.272367 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.272187 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="29c57b64-e155-4259-9599-0d6815e4dc8d" containerName="manager" Jul 08 04:27:20.278262 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.278243 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4s4z" Jul 08 04:27:20.280596 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.280575 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 08 04:27:20.285092 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.285069 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:27:20.309076 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.309053 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:27:20.347615 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.347593 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vq8l\" (UniqueName: \"kubernetes.io/projected/98a7af65-5b90-4d3a-b8b4-93a03b8187b2-kube-api-access-6vq8l\") pod \"limitador-limitador-54d6d988d5-v4s4z\" (UID: \"98a7af65-5b90-4d3a-b8b4-93a03b8187b2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4s4z" Jul 08 04:27:20.347696 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.347636 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98a7af65-5b90-4d3a-b8b4-93a03b8187b2-config-file\") pod \"limitador-limitador-54d6d988d5-v4s4z\" (UID: \"98a7af65-5b90-4d3a-b8b4-93a03b8187b2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4s4z" Jul 08 04:27:20.448045 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.448019 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vq8l\" (UniqueName: \"kubernetes.io/projected/98a7af65-5b90-4d3a-b8b4-93a03b8187b2-kube-api-access-6vq8l\") pod \"limitador-limitador-54d6d988d5-v4s4z\" (UID: \"98a7af65-5b90-4d3a-b8b4-93a03b8187b2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4s4z" Jul 08 04:27:20.448157 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.448062 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98a7af65-5b90-4d3a-b8b4-93a03b8187b2-config-file\") pod \"limitador-limitador-54d6d988d5-v4s4z\" (UID: \"98a7af65-5b90-4d3a-b8b4-93a03b8187b2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4s4z" Jul 08 04:27:20.448655 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.448640 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/98a7af65-5b90-4d3a-b8b4-93a03b8187b2-config-file\") pod \"limitador-limitador-54d6d988d5-v4s4z\" (UID: \"98a7af65-5b90-4d3a-b8b4-93a03b8187b2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4s4z" Jul 08 04:27:20.455372 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.455332 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vq8l\" (UniqueName: \"kubernetes.io/projected/98a7af65-5b90-4d3a-b8b4-93a03b8187b2-kube-api-access-6vq8l\") pod \"limitador-limitador-54d6d988d5-v4s4z\" (UID: \"98a7af65-5b90-4d3a-b8b4-93a03b8187b2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4s4z" Jul 08 04:27:20.588551 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.588492 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4s4z" Jul 08 04:27:20.710406 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.710374 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:27:20.713261 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:27:20.713235 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a7af65_5b90_4d3a_b8b4_93a03b8187b2.slice/crio-bcced1d1406f790f71bb0d4c09274f0b2874f83644ba1f5ff20c13326b0b31a9 WatchSource:0}: Error finding container bcced1d1406f790f71bb0d4c09274f0b2874f83644ba1f5ff20c13326b0b31a9: Status 404 returned error can't find the container with id bcced1d1406f790f71bb0d4c09274f0b2874f83644ba1f5ff20c13326b0b31a9 Jul 08 04:27:20.957836 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.957807 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-b9n2z"] Jul 08 04:27:20.962359 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.962328 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b9n2z" Jul 08 04:27:20.966717 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.966674 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mv9l9\"" Jul 08 04:27:20.968594 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:20.968571 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b9n2z"] Jul 08 04:27:21.052570 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:21.052539 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpnjn\" (UniqueName: \"kubernetes.io/projected/542c8054-0548-40cf-887b-605617b6ebc7-kube-api-access-fpnjn\") pod \"authorino-75d556bd77-b9n2z\" (UID: \"542c8054-0548-40cf-887b-605617b6ebc7\") " pod="kuadrant-system/authorino-75d556bd77-b9n2z" Jul 08 04:27:21.104521 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:21.104488 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4s4z" event={"ID":"98a7af65-5b90-4d3a-b8b4-93a03b8187b2","Type":"ContainerStarted","Data":"bcced1d1406f790f71bb0d4c09274f0b2874f83644ba1f5ff20c13326b0b31a9"} Jul 08 04:27:21.153776 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:21.153735 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpnjn\" (UniqueName: \"kubernetes.io/projected/542c8054-0548-40cf-887b-605617b6ebc7-kube-api-access-fpnjn\") pod \"authorino-75d556bd77-b9n2z\" (UID: \"542c8054-0548-40cf-887b-605617b6ebc7\") " pod="kuadrant-system/authorino-75d556bd77-b9n2z" Jul 08 04:27:21.164470 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:21.164447 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpnjn\" (UniqueName: \"kubernetes.io/projected/542c8054-0548-40cf-887b-605617b6ebc7-kube-api-access-fpnjn\") pod \"authorino-75d556bd77-b9n2z\" (UID: \"542c8054-0548-40cf-887b-605617b6ebc7\") " pod="kuadrant-system/authorino-75d556bd77-b9n2z" Jul 08 04:27:21.271784 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:21.271688 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b9n2z" Jul 08 04:27:21.438386 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:21.438296 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b9n2z"] Jul 08 04:27:21.442092 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:27:21.442065 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod542c8054_0548_40cf_887b_605617b6ebc7.slice/crio-039f162aad3f818fabb0d65ae60fa46a6aedde507c34c45873f6c163e6a38b11 WatchSource:0}: Error finding container 039f162aad3f818fabb0d65ae60fa46a6aedde507c34c45873f6c163e6a38b11: Status 404 returned error can't find the container with id 039f162aad3f818fabb0d65ae60fa46a6aedde507c34c45873f6c163e6a38b11 Jul 08 04:27:22.111875 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:22.111830 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b9n2z" event={"ID":"542c8054-0548-40cf-887b-605617b6ebc7","Type":"ContainerStarted","Data":"039f162aad3f818fabb0d65ae60fa46a6aedde507c34c45873f6c163e6a38b11"} Jul 08 04:27:25.131915 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:25.131873 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4s4z" event={"ID":"98a7af65-5b90-4d3a-b8b4-93a03b8187b2","Type":"ContainerStarted","Data":"00364706c0552ac9f0c0a2b5c759acf91716bb979c4ddc02a308b20f508474e5"} Jul 08 04:27:25.132393 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:25.132026 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4s4z" Jul 08 04:27:25.133609 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:25.133578 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b9n2z" event={"ID":"542c8054-0548-40cf-887b-605617b6ebc7","Type":"ContainerStarted","Data":"92a2f2003131e2dbf3b9153e75d402c01c4df3b80743673f258155ae28647508"} Jul 08 04:27:25.152047 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:25.152002 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4s4z" podStartSLOduration=0.935258744 podStartE2EDuration="5.151990388s" totalImagesPullingTime="4.216731644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:27:25.150222851 +0000 UTC m=+537.586997831" watchObservedRunningTime="2026-07-08 04:27:25.151990388 +0000 UTC m=+537.588765386" Jul 08 04:27:25.165178 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:25.165140 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-b9n2z" podStartSLOduration=1.621127524 podStartE2EDuration="5.165125345s" totalImagesPullingTime="3.543997821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:27:25.164488143 +0000 UTC m=+537.601263132" watchObservedRunningTime="2026-07-08 04:27:25.165125345 +0000 UTC m=+537.601900343" Jul 08 04:27:36.139955 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:36.139925 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4s4z" Jul 08 04:27:55.113929 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:55.113890 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76df8884d8-dtr84"] Jul 08 04:27:55.122474 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:55.122450 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76df8884d8-dtr84" Jul 08 04:27:55.123394 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:55.123372 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76df8884d8-dtr84"] Jul 08 04:27:55.125371 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:55.125334 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 08 04:27:55.238199 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:55.238164 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7b710328-9136-45cb-9b12-3f17da15613d-tls-cert\") pod \"authorino-76df8884d8-dtr84\" (UID: \"7b710328-9136-45cb-9b12-3f17da15613d\") " pod="kuadrant-system/authorino-76df8884d8-dtr84" Jul 08 04:27:55.238416 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:55.238225 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfmtb\" (UniqueName: \"kubernetes.io/projected/7b710328-9136-45cb-9b12-3f17da15613d-kube-api-access-jfmtb\") pod \"authorino-76df8884d8-dtr84\" (UID: \"7b710328-9136-45cb-9b12-3f17da15613d\") " pod="kuadrant-system/authorino-76df8884d8-dtr84" Jul 08 04:27:55.339140 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:55.339105 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7b710328-9136-45cb-9b12-3f17da15613d-tls-cert\") pod \"authorino-76df8884d8-dtr84\" (UID: \"7b710328-9136-45cb-9b12-3f17da15613d\") " pod="kuadrant-system/authorino-76df8884d8-dtr84" Jul 08 04:27:55.339312 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:55.339168 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfmtb\" (UniqueName: \"kubernetes.io/projected/7b710328-9136-45cb-9b12-3f17da15613d-kube-api-access-jfmtb\") pod \"authorino-76df8884d8-dtr84\" (UID: \"7b710328-9136-45cb-9b12-3f17da15613d\") " pod="kuadrant-system/authorino-76df8884d8-dtr84" Jul 08 04:27:55.341581 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:55.341556 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7b710328-9136-45cb-9b12-3f17da15613d-tls-cert\") pod \"authorino-76df8884d8-dtr84\" (UID: \"7b710328-9136-45cb-9b12-3f17da15613d\") " pod="kuadrant-system/authorino-76df8884d8-dtr84" Jul 08 04:27:55.347128 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:55.347099 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfmtb\" (UniqueName: \"kubernetes.io/projected/7b710328-9136-45cb-9b12-3f17da15613d-kube-api-access-jfmtb\") pod \"authorino-76df8884d8-dtr84\" (UID: \"7b710328-9136-45cb-9b12-3f17da15613d\") " pod="kuadrant-system/authorino-76df8884d8-dtr84" Jul 08 04:27:55.431895 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:55.431864 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76df8884d8-dtr84" Jul 08 04:27:55.557820 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:55.557797 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76df8884d8-dtr84"] Jul 08 04:27:55.560223 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:27:55.560188 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b710328_9136_45cb_9b12_3f17da15613d.slice/crio-79d407ae3bed860a3bcc0516ee1a1eb755aef0cf27f408645743f33724eed82a WatchSource:0}: Error finding container 79d407ae3bed860a3bcc0516ee1a1eb755aef0cf27f408645743f33724eed82a: Status 404 returned error can't find the container with id 79d407ae3bed860a3bcc0516ee1a1eb755aef0cf27f408645743f33724eed82a Jul 08 04:27:56.254711 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:56.254673 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76df8884d8-dtr84" event={"ID":"7b710328-9136-45cb-9b12-3f17da15613d","Type":"ContainerStarted","Data":"1d82e72bf712b3de7f0149cf9eb19fa504b6959b44da42febe0fbbbc3dcc5e31"} Jul 08 04:27:56.254711 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:56.254719 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76df8884d8-dtr84" event={"ID":"7b710328-9136-45cb-9b12-3f17da15613d","Type":"ContainerStarted","Data":"79d407ae3bed860a3bcc0516ee1a1eb755aef0cf27f408645743f33724eed82a"} Jul 08 04:27:56.271674 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:56.271625 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-76df8884d8-dtr84" podStartSLOduration=0.768718574 podStartE2EDuration="1.271609906s" totalImagesPullingTime="502.891332ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:27:56.270744622 +0000 UTC m=+568.707519601" watchObservedRunningTime="2026-07-08 04:27:56.271609906 +0000 UTC m=+568.708384932" Jul 08 04:27:56.297270 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:56.297236 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b9n2z"] Jul 08 04:27:56.297465 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:56.297422 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-b9n2z" podUID="542c8054-0548-40cf-887b-605617b6ebc7" containerName="authorino" containerID="cri-o://92a2f2003131e2dbf3b9153e75d402c01c4df3b80743673f258155ae28647508" gracePeriod=30 Jul 08 04:27:56.536409 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:56.536378 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b9n2z" Jul 08 04:27:56.649692 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:56.649661 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpnjn\" (UniqueName: \"kubernetes.io/projected/542c8054-0548-40cf-887b-605617b6ebc7-kube-api-access-fpnjn\") pod \"542c8054-0548-40cf-887b-605617b6ebc7\" (UID: \"542c8054-0548-40cf-887b-605617b6ebc7\") " Jul 08 04:27:56.651730 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:56.651703 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/542c8054-0548-40cf-887b-605617b6ebc7-kube-api-access-fpnjn" (OuterVolumeSpecName: "kube-api-access-fpnjn") pod "542c8054-0548-40cf-887b-605617b6ebc7" (UID: "542c8054-0548-40cf-887b-605617b6ebc7"). InnerVolumeSpecName "kube-api-access-fpnjn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:27:56.751152 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:56.751116 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpnjn\" (UniqueName: \"kubernetes.io/projected/542c8054-0548-40cf-887b-605617b6ebc7-kube-api-access-fpnjn\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:27:57.259951 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:57.259914 2583 generic.go:356] "Generic (PLEG): container finished" podID="542c8054-0548-40cf-887b-605617b6ebc7" containerID="92a2f2003131e2dbf3b9153e75d402c01c4df3b80743673f258155ae28647508" exitCode=0 Jul 08 04:27:57.260448 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:57.259960 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b9n2z" Jul 08 04:27:57.260448 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:57.260000 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b9n2z" event={"ID":"542c8054-0548-40cf-887b-605617b6ebc7","Type":"ContainerDied","Data":"92a2f2003131e2dbf3b9153e75d402c01c4df3b80743673f258155ae28647508"} Jul 08 04:27:57.260448 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:57.260043 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b9n2z" event={"ID":"542c8054-0548-40cf-887b-605617b6ebc7","Type":"ContainerDied","Data":"039f162aad3f818fabb0d65ae60fa46a6aedde507c34c45873f6c163e6a38b11"} Jul 08 04:27:57.260448 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:57.260059 2583 scope.go:117] "RemoveContainer" containerID="92a2f2003131e2dbf3b9153e75d402c01c4df3b80743673f258155ae28647508" Jul 08 04:27:57.267918 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:57.267901 2583 scope.go:117] "RemoveContainer" containerID="92a2f2003131e2dbf3b9153e75d402c01c4df3b80743673f258155ae28647508" Jul 08 04:27:57.268158 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:27:57.268140 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92a2f2003131e2dbf3b9153e75d402c01c4df3b80743673f258155ae28647508\": container with ID starting with 92a2f2003131e2dbf3b9153e75d402c01c4df3b80743673f258155ae28647508 not found: ID does not exist" containerID="92a2f2003131e2dbf3b9153e75d402c01c4df3b80743673f258155ae28647508" Jul 08 04:27:57.268203 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:57.268166 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92a2f2003131e2dbf3b9153e75d402c01c4df3b80743673f258155ae28647508"} err="failed to get container status \"92a2f2003131e2dbf3b9153e75d402c01c4df3b80743673f258155ae28647508\": rpc error: code = NotFound desc = could not find container \"92a2f2003131e2dbf3b9153e75d402c01c4df3b80743673f258155ae28647508\": container with ID starting with 92a2f2003131e2dbf3b9153e75d402c01c4df3b80743673f258155ae28647508 not found: ID does not exist" Jul 08 04:27:57.279969 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:57.279949 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b9n2z"] Jul 08 04:27:57.282061 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:57.282043 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b9n2z"] Jul 08 04:27:58.073890 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:27:58.073860 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="542c8054-0548-40cf-887b-605617b6ebc7" path="/var/lib/kubelet/pods/542c8054-0548-40cf-887b-605617b6ebc7/volumes" Jul 08 04:28:27.909542 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:28:27.909514 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:28:27.909995 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:28:27.909901 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:28:34.611757 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:28:34.611724 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:28:50.932004 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:28:50.931920 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:29:05.826198 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:05.826162 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:29:10.634207 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:10.634163 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:29:16.383879 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.383847 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2"] Jul 08 04:29:16.384252 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.384150 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="542c8054-0548-40cf-887b-605617b6ebc7" containerName="authorino" Jul 08 04:29:16.384252 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.384159 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="542c8054-0548-40cf-887b-605617b6ebc7" containerName="authorino" Jul 08 04:29:16.384252 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.384221 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="542c8054-0548-40cf-887b-605617b6ebc7" containerName="authorino" Jul 08 04:29:16.387165 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.387150 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.390949 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.390928 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 04:29:16.391076 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.391038 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 04:29:16.391076 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.391056 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zrrld\"" Jul 08 04:29:16.391183 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.391088 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 08 04:29:16.402110 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.402089 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2"] Jul 08 04:29:16.469781 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.469746 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e2b3851-0b99-4e9a-8bdc-446f59239f35-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.469781 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.469782 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e2b3851-0b99-4e9a-8bdc-446f59239f35-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.469989 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.469918 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e2b3851-0b99-4e9a-8bdc-446f59239f35-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.469989 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.469963 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e2b3851-0b99-4e9a-8bdc-446f59239f35-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.470082 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.470055 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nnvk\" (UniqueName: \"kubernetes.io/projected/6e2b3851-0b99-4e9a-8bdc-446f59239f35-kube-api-access-5nnvk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.470127 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.470098 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e2b3851-0b99-4e9a-8bdc-446f59239f35-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.571204 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.571167 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e2b3851-0b99-4e9a-8bdc-446f59239f35-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.571204 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.571210 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e2b3851-0b99-4e9a-8bdc-446f59239f35-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.571461 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.571249 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nnvk\" (UniqueName: \"kubernetes.io/projected/6e2b3851-0b99-4e9a-8bdc-446f59239f35-kube-api-access-5nnvk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.571461 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.571275 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e2b3851-0b99-4e9a-8bdc-446f59239f35-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.571461 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.571319 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e2b3851-0b99-4e9a-8bdc-446f59239f35-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.571614 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.571539 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e2b3851-0b99-4e9a-8bdc-446f59239f35-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.571733 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.571701 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e2b3851-0b99-4e9a-8bdc-446f59239f35-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.571866 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.571754 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e2b3851-0b99-4e9a-8bdc-446f59239f35-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.571866 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.571812 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e2b3851-0b99-4e9a-8bdc-446f59239f35-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.573609 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.573589 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e2b3851-0b99-4e9a-8bdc-446f59239f35-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.573758 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.573742 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e2b3851-0b99-4e9a-8bdc-446f59239f35-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.579443 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.579421 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nnvk\" (UniqueName: \"kubernetes.io/projected/6e2b3851-0b99-4e9a-8bdc-446f59239f35-kube-api-access-5nnvk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2\" (UID: \"6e2b3851-0b99-4e9a-8bdc-446f59239f35\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.696506 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.696474 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:16.820247 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.820225 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2"] Jul 08 04:29:16.822297 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:29:16.822272 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e2b3851_0b99_4e9a_8bdc_446f59239f35.slice/crio-e9bc25844c8bdf95be3c7fd4cecc418eb180313e5a448ff53ef4e8c888d553c5 WatchSource:0}: Error finding container e9bc25844c8bdf95be3c7fd4cecc418eb180313e5a448ff53ef4e8c888d553c5: Status 404 returned error can't find the container with id e9bc25844c8bdf95be3c7fd4cecc418eb180313e5a448ff53ef4e8c888d553c5 Jul 08 04:29:16.823575 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:16.823559 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 04:29:17.584807 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:17.584770 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" event={"ID":"6e2b3851-0b99-4e9a-8bdc-446f59239f35","Type":"ContainerStarted","Data":"e9bc25844c8bdf95be3c7fd4cecc418eb180313e5a448ff53ef4e8c888d553c5"} Jul 08 04:29:18.237354 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:18.237321 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:29:18.591036 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:18.590943 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" event={"ID":"6e2b3851-0b99-4e9a-8bdc-446f59239f35","Type":"ContainerStarted","Data":"cf50c99946477bbfd75a07fc11237f5b5218e8fa46acfb1eae070cb128714597"} Jul 08 04:29:18.609111 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:18.609052 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" podStartSLOduration=1.61153674 podStartE2EDuration="2.609033131s" totalImagesPullingTime="997.496391ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:29:18.607860241 +0000 UTC m=+651.044635220" watchObservedRunningTime="2026-07-08 04:29:18.609033131 +0000 UTC m=+651.045808110" Jul 08 04:29:26.697375 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:26.697322 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:26.697754 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:26.697407 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:26.709957 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:26.709933 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:27.639571 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:27.639545 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2" Jul 08 04:29:41.138511 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:41.138475 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:29:47.783621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:47.783590 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh"] Jul 08 04:29:47.789941 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:47.789922 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:47.792481 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:47.792458 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 08 04:29:47.800728 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:47.800707 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh"] Jul 08 04:29:47.929547 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:47.929515 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dd00104-f6d8-4400-9658-b85abef5b854-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:47.929547 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:47.929546 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dd00104-f6d8-4400-9658-b85abef5b854-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:47.929773 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:47.929590 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd00104-f6d8-4400-9658-b85abef5b854-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:47.929773 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:47.929663 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq8qq\" (UniqueName: \"kubernetes.io/projected/8dd00104-f6d8-4400-9658-b85abef5b854-kube-api-access-qq8qq\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:47.929773 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:47.929715 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dd00104-f6d8-4400-9658-b85abef5b854-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:47.929773 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:47.929742 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dd00104-f6d8-4400-9658-b85abef5b854-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:48.031004 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.030969 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd00104-f6d8-4400-9658-b85abef5b854-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:48.031182 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.031022 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq8qq\" (UniqueName: \"kubernetes.io/projected/8dd00104-f6d8-4400-9658-b85abef5b854-kube-api-access-qq8qq\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:48.031182 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.031058 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dd00104-f6d8-4400-9658-b85abef5b854-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:48.031182 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.031083 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dd00104-f6d8-4400-9658-b85abef5b854-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:48.031182 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.031111 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dd00104-f6d8-4400-9658-b85abef5b854-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:48.031182 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.031137 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dd00104-f6d8-4400-9658-b85abef5b854-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:48.031626 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.031598 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dd00104-f6d8-4400-9658-b85abef5b854-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:48.031755 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.031632 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dd00104-f6d8-4400-9658-b85abef5b854-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:48.031755 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.031658 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dd00104-f6d8-4400-9658-b85abef5b854-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:48.033305 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.033279 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dd00104-f6d8-4400-9658-b85abef5b854-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:48.033580 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.033564 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd00104-f6d8-4400-9658-b85abef5b854-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:48.038564 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.038510 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq8qq\" (UniqueName: \"kubernetes.io/projected/8dd00104-f6d8-4400-9658-b85abef5b854-kube-api-access-qq8qq\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh\" (UID: \"8dd00104-f6d8-4400-9658-b85abef5b854\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:48.098794 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.098767 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:48.225369 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.225328 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh"] Jul 08 04:29:48.227734 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:29:48.227703 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8dd00104_f6d8_4400_9658_b85abef5b854.slice/crio-bde52b5c43bdc49190782e5bf3a857fc50c562f8400879795b43794db672baf6 WatchSource:0}: Error finding container bde52b5c43bdc49190782e5bf3a857fc50c562f8400879795b43794db672baf6: Status 404 returned error can't find the container with id bde52b5c43bdc49190782e5bf3a857fc50c562f8400879795b43794db672baf6 Jul 08 04:29:48.714085 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.714004 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" event={"ID":"8dd00104-f6d8-4400-9658-b85abef5b854","Type":"ContainerStarted","Data":"18591a41210f7e061b8cbf41a34a3e900c745006edfde479e65e332792bcebb6"} Jul 08 04:29:48.714085 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.714040 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" event={"ID":"8dd00104-f6d8-4400-9658-b85abef5b854","Type":"ContainerStarted","Data":"bde52b5c43bdc49190782e5bf3a857fc50c562f8400879795b43794db672baf6"} Jul 08 04:29:48.731324 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:48.731274 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" podStartSLOduration=1.556565376 podStartE2EDuration="1.731258806s" totalImagesPullingTime="174.69343ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:29:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:29:48.729941139 +0000 UTC m=+681.166716126" watchObservedRunningTime="2026-07-08 04:29:48.731258806 +0000 UTC m=+681.168033784" Jul 08 04:29:50.330277 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:50.330241 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:29:58.099786 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:58.099754 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:58.099786 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:58.099794 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:58.111962 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:58.111940 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:29:58.768077 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:29:58.768049 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh" Jul 08 04:30:41.951042 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:41.951001 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f868c74df-9d9js"] Jul 08 04:30:41.954502 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:41.954479 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f868c74df-9d9js" Jul 08 04:30:41.960748 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:41.960721 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f868c74df-9d9js"] Jul 08 04:30:42.078076 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:42.078045 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d98f2f76-390d-425e-b338-2cc0decfa194-tls-cert\") pod \"authorino-f868c74df-9d9js\" (UID: \"d98f2f76-390d-425e-b338-2cc0decfa194\") " pod="kuadrant-system/authorino-f868c74df-9d9js" Jul 08 04:30:42.078267 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:42.078090 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzzmt\" (UniqueName: \"kubernetes.io/projected/d98f2f76-390d-425e-b338-2cc0decfa194-kube-api-access-mzzmt\") pod \"authorino-f868c74df-9d9js\" (UID: \"d98f2f76-390d-425e-b338-2cc0decfa194\") " pod="kuadrant-system/authorino-f868c74df-9d9js" Jul 08 04:30:42.179055 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:42.179014 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d98f2f76-390d-425e-b338-2cc0decfa194-tls-cert\") pod \"authorino-f868c74df-9d9js\" (UID: \"d98f2f76-390d-425e-b338-2cc0decfa194\") " pod="kuadrant-system/authorino-f868c74df-9d9js" Jul 08 04:30:42.179055 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:42.179056 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzzmt\" (UniqueName: \"kubernetes.io/projected/d98f2f76-390d-425e-b338-2cc0decfa194-kube-api-access-mzzmt\") pod \"authorino-f868c74df-9d9js\" (UID: \"d98f2f76-390d-425e-b338-2cc0decfa194\") " pod="kuadrant-system/authorino-f868c74df-9d9js" Jul 08 04:30:42.181506 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:42.181482 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d98f2f76-390d-425e-b338-2cc0decfa194-tls-cert\") pod \"authorino-f868c74df-9d9js\" (UID: \"d98f2f76-390d-425e-b338-2cc0decfa194\") " pod="kuadrant-system/authorino-f868c74df-9d9js" Jul 08 04:30:42.186655 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:42.186637 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzzmt\" (UniqueName: \"kubernetes.io/projected/d98f2f76-390d-425e-b338-2cc0decfa194-kube-api-access-mzzmt\") pod \"authorino-f868c74df-9d9js\" (UID: \"d98f2f76-390d-425e-b338-2cc0decfa194\") " pod="kuadrant-system/authorino-f868c74df-9d9js" Jul 08 04:30:42.264785 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:42.264700 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f868c74df-9d9js" Jul 08 04:30:42.397780 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:42.397755 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f868c74df-9d9js"] Jul 08 04:30:42.399303 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:30:42.399278 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd98f2f76_390d_425e_b338_2cc0decfa194.slice/crio-fbf0561f4170203d046e7edb54cf945995b9a69ea2054adbcdd6f83e639f8e0a WatchSource:0}: Error finding container fbf0561f4170203d046e7edb54cf945995b9a69ea2054adbcdd6f83e639f8e0a: Status 404 returned error can't find the container with id fbf0561f4170203d046e7edb54cf945995b9a69ea2054adbcdd6f83e639f8e0a Jul 08 04:30:42.941017 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:42.940980 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f868c74df-9d9js" event={"ID":"d98f2f76-390d-425e-b338-2cc0decfa194","Type":"ContainerStarted","Data":"fbf0561f4170203d046e7edb54cf945995b9a69ea2054adbcdd6f83e639f8e0a"} Jul 08 04:30:43.948531 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:43.948496 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f868c74df-9d9js" event={"ID":"d98f2f76-390d-425e-b338-2cc0decfa194","Type":"ContainerStarted","Data":"943e251e0e2f8cff0121e00883e8df86a67f961d006376d81713ab6ce97038a4"} Jul 08 04:30:44.066963 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.066894 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-f868c74df-9d9js" podStartSLOduration=2.549446038 podStartE2EDuration="3.066870126s" totalImagesPullingTime="517.424088ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:30:43.984334255 +0000 UTC m=+736.421109234" watchObservedRunningTime="2026-07-08 04:30:44.066870126 +0000 UTC m=+736.503645106" Jul 08 04:30:44.067308 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.067289 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76df8884d8-dtr84"] Jul 08 04:30:44.067552 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.067518 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-76df8884d8-dtr84" podUID="7b710328-9136-45cb-9b12-3f17da15613d" containerName="authorino" containerID="cri-o://1d82e72bf712b3de7f0149cf9eb19fa504b6959b44da42febe0fbbbc3dcc5e31" gracePeriod=30 Jul 08 04:30:44.319505 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.319479 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76df8884d8-dtr84" Jul 08 04:30:44.402276 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.402244 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfmtb\" (UniqueName: \"kubernetes.io/projected/7b710328-9136-45cb-9b12-3f17da15613d-kube-api-access-jfmtb\") pod \"7b710328-9136-45cb-9b12-3f17da15613d\" (UID: \"7b710328-9136-45cb-9b12-3f17da15613d\") " Jul 08 04:30:44.402276 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.402284 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7b710328-9136-45cb-9b12-3f17da15613d-tls-cert\") pod \"7b710328-9136-45cb-9b12-3f17da15613d\" (UID: \"7b710328-9136-45cb-9b12-3f17da15613d\") " Jul 08 04:30:44.404395 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.404357 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b710328-9136-45cb-9b12-3f17da15613d-kube-api-access-jfmtb" (OuterVolumeSpecName: "kube-api-access-jfmtb") pod "7b710328-9136-45cb-9b12-3f17da15613d" (UID: "7b710328-9136-45cb-9b12-3f17da15613d"). InnerVolumeSpecName "kube-api-access-jfmtb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:30:44.411627 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.411597 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b710328-9136-45cb-9b12-3f17da15613d-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "7b710328-9136-45cb-9b12-3f17da15613d" (UID: "7b710328-9136-45cb-9b12-3f17da15613d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:30:44.503319 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.503232 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfmtb\" (UniqueName: \"kubernetes.io/projected/7b710328-9136-45cb-9b12-3f17da15613d-kube-api-access-jfmtb\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:30:44.503319 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.503262 2583 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7b710328-9136-45cb-9b12-3f17da15613d-tls-cert\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:30:44.953516 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.953484 2583 generic.go:356] "Generic (PLEG): container finished" podID="7b710328-9136-45cb-9b12-3f17da15613d" containerID="1d82e72bf712b3de7f0149cf9eb19fa504b6959b44da42febe0fbbbc3dcc5e31" exitCode=0 Jul 08 04:30:44.953983 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.953527 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76df8884d8-dtr84" Jul 08 04:30:44.953983 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.953573 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76df8884d8-dtr84" event={"ID":"7b710328-9136-45cb-9b12-3f17da15613d","Type":"ContainerDied","Data":"1d82e72bf712b3de7f0149cf9eb19fa504b6959b44da42febe0fbbbc3dcc5e31"} Jul 08 04:30:44.953983 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.953619 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76df8884d8-dtr84" event={"ID":"7b710328-9136-45cb-9b12-3f17da15613d","Type":"ContainerDied","Data":"79d407ae3bed860a3bcc0516ee1a1eb755aef0cf27f408645743f33724eed82a"} Jul 08 04:30:44.953983 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.953641 2583 scope.go:117] "RemoveContainer" containerID="1d82e72bf712b3de7f0149cf9eb19fa504b6959b44da42febe0fbbbc3dcc5e31" Jul 08 04:30:44.962767 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.962750 2583 scope.go:117] "RemoveContainer" containerID="1d82e72bf712b3de7f0149cf9eb19fa504b6959b44da42febe0fbbbc3dcc5e31" Jul 08 04:30:44.963015 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:30:44.962997 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d82e72bf712b3de7f0149cf9eb19fa504b6959b44da42febe0fbbbc3dcc5e31\": container with ID starting with 1d82e72bf712b3de7f0149cf9eb19fa504b6959b44da42febe0fbbbc3dcc5e31 not found: ID does not exist" containerID="1d82e72bf712b3de7f0149cf9eb19fa504b6959b44da42febe0fbbbc3dcc5e31" Jul 08 04:30:44.963076 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.963026 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d82e72bf712b3de7f0149cf9eb19fa504b6959b44da42febe0fbbbc3dcc5e31"} err="failed to get container status \"1d82e72bf712b3de7f0149cf9eb19fa504b6959b44da42febe0fbbbc3dcc5e31\": rpc error: code = NotFound desc = could not find container \"1d82e72bf712b3de7f0149cf9eb19fa504b6959b44da42febe0fbbbc3dcc5e31\": container with ID starting with 1d82e72bf712b3de7f0149cf9eb19fa504b6959b44da42febe0fbbbc3dcc5e31 not found: ID does not exist" Jul 08 04:30:44.976843 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.976818 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76df8884d8-dtr84"] Jul 08 04:30:44.982883 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:44.982861 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76df8884d8-dtr84"] Jul 08 04:30:46.073061 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:30:46.073030 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b710328-9136-45cb-9b12-3f17da15613d" path="/var/lib/kubelet/pods/7b710328-9136-45cb-9b12-3f17da15613d/volumes" Jul 08 04:31:10.837645 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:31:10.837610 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:31:19.926256 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:31:19.926214 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:31:32.323302 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:31:32.323269 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:31:40.135669 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:31:40.135631 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:31:51.631284 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:31:51.631199 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:32:18.633485 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:32:18.633454 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:32:32.231969 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:32:32.231935 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:33:05.925389 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:33:05.925353 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:33:15.326166 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:33:15.326129 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:33:25.830846 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:33:25.830810 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:33:27.936029 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:33:27.936003 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:33:27.936729 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:33:27.936711 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:33:33.124757 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:33:33.124721 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:33:44.731457 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:33:44.731416 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:33:54.429484 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:33:54.429447 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:34:05.220783 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:34:05.220750 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:34:15.631134 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:34:15.631096 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:34:37.633865 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:34:37.633827 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:34:49.635532 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:34:49.635499 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:35:13.023661 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:35:13.023625 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:35:24.134294 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:35:24.134262 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:35:40.327015 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:35:40.326974 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:35:51.736687 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:35:51.736649 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:36:01.530225 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:36:01.530189 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:36:14.028756 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:36:14.028719 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:38:07.434721 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:38:07.434641 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:38:19.630332 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:38:19.630297 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:38:27.958956 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:38:27.958929 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:38:27.961049 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:38:27.961031 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:38:28.831431 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:38:28.831399 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:38:40.937823 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:38:40.937790 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:38:53.130702 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:38:53.130662 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:39:03.131982 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:39:03.131947 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:39:11.037377 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:39:11.037328 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:39:21.641614 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:39:21.641580 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:39:32.126678 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:39:32.126639 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:39:41.729965 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:39:41.729926 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:39:52.637895 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:39:52.637854 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:40:17.333068 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:40:17.333028 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:40:24.834440 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:40:24.834405 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:40:34.426366 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:40:34.426319 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:40:43.630231 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:40:43.630194 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:40:50.326445 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:40:50.326355 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:41:16.432790 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:41:16.432759 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:41:27.730482 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:41:27.730446 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:41:41.430668 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:41:41.430635 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:41:48.777006 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:41:48.776974 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:41:58.332132 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:41:58.332088 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:42:11.128516 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:42:11.128485 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:42:20.442042 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:42:20.441956 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:42:29.937427 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:42:29.937395 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:42:40.927778 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:42:40.927744 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:42:53.535167 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:42:53.535123 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:43:02.431178 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:43:02.431144 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:43:14.136497 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:43:14.136460 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:43:24.930705 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:43:24.930669 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:43:27.989677 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:43:27.989646 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:43:27.991504 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:43:27.991485 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:43:36.631517 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:43:36.631485 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:43:51.027233 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:43:51.027135 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:44:00.129199 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:44:00.129163 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:44:10.965564 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:44:10.965528 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:44:29.926972 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:44:29.926882 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:44:40.428415 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:44:40.428377 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:44:50.523102 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:44:50.523067 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:44:57.238500 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:44:57.238467 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:46:55.227604 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:46:55.227527 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:47:04.434703 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:47:04.434669 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:48:23.540677 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:48:23.540637 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:48:28.014415 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:48:28.014387 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:48:28.016689 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:48:28.016668 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:48:33.831083 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:48:33.831047 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:48:43.642657 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:48:43.642620 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:49:10.637402 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:10.637369 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:49:11.092403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.092368 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg"] Jul 08 04:49:11.094570 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.092662 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b710328-9136-45cb-9b12-3f17da15613d" containerName="authorino" Jul 08 04:49:11.094570 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.092672 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b710328-9136-45cb-9b12-3f17da15613d" containerName="authorino" Jul 08 04:49:11.094570 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.092725 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b710328-9136-45cb-9b12-3f17da15613d" containerName="authorino" Jul 08 04:49:11.095474 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.095458 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.099285 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.099256 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fd2bd8d4\"/\"e2e-lifecycle-model-fd2bd8d4-kserve-self-signed-certs\"" Jul 08 04:49:11.099418 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.099283 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fd2bd8d4\"/\"openshift-service-ca.crt\"" Jul 08 04:49:11.099418 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.099286 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fd2bd8d4\"/\"kube-root-ca.crt\"" Jul 08 04:49:11.099418 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.099286 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fd2bd8d4\"/\"default-dockercfg-9zh4d\"" Jul 08 04:49:11.105635 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.105614 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg"] Jul 08 04:49:11.160620 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.160589 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-model-cache\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.160769 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.160623 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54cd48bb-469f-445a-a631-647d28657a96-tls-certs\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.160769 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.160681 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-dshm\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.160769 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.160716 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkqpk\" (UniqueName: \"kubernetes.io/projected/54cd48bb-469f-445a-a631-647d28657a96-kube-api-access-hkqpk\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.160908 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.160766 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-home\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.160908 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.160850 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-tmp-dir\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.261652 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.261616 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-model-cache\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.261652 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.261653 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54cd48bb-469f-445a-a631-647d28657a96-tls-certs\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.261879 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.261703 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-dshm\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.261879 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.261727 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkqpk\" (UniqueName: \"kubernetes.io/projected/54cd48bb-469f-445a-a631-647d28657a96-kube-api-access-hkqpk\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.261879 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.261748 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-home\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.261879 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.261791 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-tmp-dir\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.262086 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.262044 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-model-cache\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.262142 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.262118 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-home\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.262201 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.262174 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-tmp-dir\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.263929 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.263903 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-dshm\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.264169 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.264153 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54cd48bb-469f-445a-a631-647d28657a96-tls-certs\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.271024 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.270998 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkqpk\" (UniqueName: \"kubernetes.io/projected/54cd48bb-469f-445a-a631-647d28657a96-kube-api-access-hkqpk\") pod \"e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.404459 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.404429 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:11.531738 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.531711 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg"] Jul 08 04:49:11.534410 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:49:11.534386 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod54cd48bb_469f_445a_a631_647d28657a96.slice/crio-ae4ff69e0b94ca4d94ec62b8549f7906e37319340c411121ac6ee05d03e5dc90 WatchSource:0}: Error finding container ae4ff69e0b94ca4d94ec62b8549f7906e37319340c411121ac6ee05d03e5dc90: Status 404 returned error can't find the container with id ae4ff69e0b94ca4d94ec62b8549f7906e37319340c411121ac6ee05d03e5dc90 Jul 08 04:49:11.618848 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.618813 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" event={"ID":"54cd48bb-469f-445a-a631-647d28657a96","Type":"ContainerStarted","Data":"9c2c7d51632819a4014350ea2988a78f9283fa17c1964136e83c3b32e62c5551"} Jul 08 04:49:11.618975 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.618862 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" event={"ID":"54cd48bb-469f-445a-a631-647d28657a96","Type":"ContainerStarted","Data":"ae4ff69e0b94ca4d94ec62b8549f7906e37319340c411121ac6ee05d03e5dc90"} Jul 08 04:49:11.638047 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:11.638003 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" podStartSLOduration=0.637983389 podStartE2EDuration="637.983389ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:49:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:49:11.63746316 +0000 UTC m=+1844.074238139" watchObservedRunningTime="2026-07-08 04:49:11.637983389 +0000 UTC m=+1844.074758369" Jul 08 04:49:19.533822 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:19.533785 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:49:21.405203 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:21.405169 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:21.405712 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:21.405492 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:21.417725 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:21.417704 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:21.671813 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:21.671732 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:27.931116 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:27.931087 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:49:40.467568 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:40.467533 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:49:41.643047 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:41.643011 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg"] Jul 08 04:49:56.663151 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.663069 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" podUID="54cd48bb-469f-445a-a631-647d28657a96" containerName="main" containerID="cri-o://9c2c7d51632819a4014350ea2988a78f9283fa17c1964136e83c3b32e62c5551" gracePeriod=45 Jul 08 04:49:56.807769 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.807738 2583 generic.go:356] "Generic (PLEG): container finished" podID="54cd48bb-469f-445a-a631-647d28657a96" containerID="9c2c7d51632819a4014350ea2988a78f9283fa17c1964136e83c3b32e62c5551" exitCode=0 Jul 08 04:49:56.807950 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.807807 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" event={"ID":"54cd48bb-469f-445a-a631-647d28657a96","Type":"ContainerDied","Data":"9c2c7d51632819a4014350ea2988a78f9283fa17c1964136e83c3b32e62c5551"} Jul 08 04:49:56.908120 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.908099 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:56.945746 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.945649 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkqpk\" (UniqueName: \"kubernetes.io/projected/54cd48bb-469f-445a-a631-647d28657a96-kube-api-access-hkqpk\") pod \"54cd48bb-469f-445a-a631-647d28657a96\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " Jul 08 04:49:56.945746 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.945698 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-tmp-dir\") pod \"54cd48bb-469f-445a-a631-647d28657a96\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " Jul 08 04:49:56.945746 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.945739 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-home\") pod \"54cd48bb-469f-445a-a631-647d28657a96\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " Jul 08 04:49:56.945983 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.945768 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54cd48bb-469f-445a-a631-647d28657a96-tls-certs\") pod \"54cd48bb-469f-445a-a631-647d28657a96\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " Jul 08 04:49:56.945983 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.945791 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-dshm\") pod \"54cd48bb-469f-445a-a631-647d28657a96\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " Jul 08 04:49:56.945983 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.945868 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-model-cache\") pod \"54cd48bb-469f-445a-a631-647d28657a96\" (UID: \"54cd48bb-469f-445a-a631-647d28657a96\") " Jul 08 04:49:56.946130 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.946016 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "54cd48bb-469f-445a-a631-647d28657a96" (UID: "54cd48bb-469f-445a-a631-647d28657a96"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:49:56.946130 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.946081 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-home" (OuterVolumeSpecName: "home") pod "54cd48bb-469f-445a-a631-647d28657a96" (UID: "54cd48bb-469f-445a-a631-647d28657a96"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:49:56.946237 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.946198 2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-tmp-dir\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:49:56.946237 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.946205 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-model-cache" (OuterVolumeSpecName: "model-cache") pod "54cd48bb-469f-445a-a631-647d28657a96" (UID: "54cd48bb-469f-445a-a631-647d28657a96"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:49:56.946237 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.946214 2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-home\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:49:56.948085 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.948056 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-dshm" (OuterVolumeSpecName: "dshm") pod "54cd48bb-469f-445a-a631-647d28657a96" (UID: "54cd48bb-469f-445a-a631-647d28657a96"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:49:56.948263 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.948235 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54cd48bb-469f-445a-a631-647d28657a96-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "54cd48bb-469f-445a-a631-647d28657a96" (UID: "54cd48bb-469f-445a-a631-647d28657a96"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:49:56.948311 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:56.948246 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54cd48bb-469f-445a-a631-647d28657a96-kube-api-access-hkqpk" (OuterVolumeSpecName: "kube-api-access-hkqpk") pod "54cd48bb-469f-445a-a631-647d28657a96" (UID: "54cd48bb-469f-445a-a631-647d28657a96"). InnerVolumeSpecName "kube-api-access-hkqpk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:49:57.047094 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:57.047049 2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-model-cache\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:49:57.047094 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:57.047088 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkqpk\" (UniqueName: \"kubernetes.io/projected/54cd48bb-469f-445a-a631-647d28657a96-kube-api-access-hkqpk\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:49:57.047094 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:57.047100 2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54cd48bb-469f-445a-a631-647d28657a96-tls-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:49:57.047094 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:57.047109 2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54cd48bb-469f-445a-a631-647d28657a96-dshm\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:49:57.812767 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:57.812735 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" event={"ID":"54cd48bb-469f-445a-a631-647d28657a96","Type":"ContainerDied","Data":"ae4ff69e0b94ca4d94ec62b8549f7906e37319340c411121ac6ee05d03e5dc90"} Jul 08 04:49:57.812767 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:57.812755 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg" Jul 08 04:49:57.813247 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:57.812786 2583 scope.go:117] "RemoveContainer" containerID="9c2c7d51632819a4014350ea2988a78f9283fa17c1964136e83c3b32e62c5551" Jul 08 04:49:57.835393 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:57.835369 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg"] Jul 08 04:49:57.840383 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:57.840330 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-fd2bd8d4/e2e-lifecycle-model-fd2bd8d4-kserve-6ff98b76cf-zr6fg"] Jul 08 04:49:58.073836 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:49:58.073761 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54cd48bb-469f-445a-a631-647d28657a96" path="/var/lib/kubelet/pods/54cd48bb-469f-445a-a631-647d28657a96/volumes" Jul 08 04:50:21.334242 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:50:21.334205 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:50:36.033906 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:50:36.033868 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:50:46.428893 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:50:46.428857 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:51:53.731462 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:51:53.731428 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:52:51.131708 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:52:51.131672 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:53:03.436757 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:03.436725 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:53:12.393717 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.393688 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727"] Jul 08 04:53:12.394072 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.393985 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54cd48bb-469f-445a-a631-647d28657a96" containerName="main" Jul 08 04:53:12.394072 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.393995 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="54cd48bb-469f-445a-a631-647d28657a96" containerName="main" Jul 08 04:53:12.394140 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.394072 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="54cd48bb-469f-445a-a631-647d28657a96" containerName="main" Jul 08 04:53:12.396974 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.396949 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.400627 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.400606 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-f87c81\"/\"openshift-service-ca.crt\"" Jul 08 04:53:12.400627 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.400627 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-f87c81\"/\"kube-root-ca.crt\"" Jul 08 04:53:12.400833 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.400606 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-f87c81\"/\"default-dockercfg-kskfz\"" Jul 08 04:53:12.400833 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.400604 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-f87c81\"/\"test-model-f87c81-kserve-self-signed-certs\"" Jul 08 04:53:12.407783 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.407765 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727"] Jul 08 04:53:12.519931 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.519897 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-dshm\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.520088 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.519939 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-model-cache\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.520088 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.520000 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-tmp-dir\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.520088 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.520035 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/372e6930-a79a-40c0-b412-148d013047d8-tls-certs\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.520088 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.520068 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4plg\" (UniqueName: \"kubernetes.io/projected/372e6930-a79a-40c0-b412-148d013047d8-kube-api-access-l4plg\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.520293 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.520122 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-home\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.620997 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.620965 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-model-cache\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.621157 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.621015 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-tmp-dir\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.621157 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.621041 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/372e6930-a79a-40c0-b412-148d013047d8-tls-certs\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.621157 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.621071 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4plg\" (UniqueName: \"kubernetes.io/projected/372e6930-a79a-40c0-b412-148d013047d8-kube-api-access-l4plg\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.621157 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.621118 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-home\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.621425 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.621166 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-dshm\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.621497 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.621434 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-model-cache\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.621497 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.621464 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-tmp-dir\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.621573 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.621497 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-home\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.623358 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.623321 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-dshm\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.623572 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.623555 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/372e6930-a79a-40c0-b412-148d013047d8-tls-certs\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.629204 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.629183 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4plg\" (UniqueName: \"kubernetes.io/projected/372e6930-a79a-40c0-b412-148d013047d8-kube-api-access-l4plg\") pod \"test-model-f87c81-kserve-5bb8587ff-fr727\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.705674 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.705653 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:12.839508 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:12.839485 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727"] Jul 08 04:53:12.841964 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:53:12.841936 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod372e6930_a79a_40c0_b412_148d013047d8.slice/crio-dc5c50afd6b48c54b62ce262943eb68ded5e587c4257ae110df9101be5639820 WatchSource:0}: Error finding container dc5c50afd6b48c54b62ce262943eb68ded5e587c4257ae110df9101be5639820: Status 404 returned error can't find the container with id dc5c50afd6b48c54b62ce262943eb68ded5e587c4257ae110df9101be5639820 Jul 08 04:53:13.604877 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:13.604839 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" event={"ID":"372e6930-a79a-40c0-b412-148d013047d8","Type":"ContainerStarted","Data":"a66b173971077ebe675808ebbfb494b63d41f9754560cb4aeee8d91f6f8829b1"} Jul 08 04:53:13.605257 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:13.604888 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" event={"ID":"372e6930-a79a-40c0-b412-148d013047d8","Type":"ContainerStarted","Data":"dc5c50afd6b48c54b62ce262943eb68ded5e587c4257ae110df9101be5639820"} Jul 08 04:53:13.622067 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:13.622022 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" podStartSLOduration=1.622007331 podStartE2EDuration="1.622007331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:53:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:53:13.620554621 +0000 UTC m=+2086.057329614" watchObservedRunningTime="2026-07-08 04:53:13.622007331 +0000 UTC m=+2086.058782309" Jul 08 04:53:15.830596 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:15.830559 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:53:22.706177 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:22.706148 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:22.706699 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:22.706193 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:22.719307 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:22.719284 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:23.659580 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:23.659543 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:53:26.829403 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:26.829369 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:53:28.038959 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:28.038922 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:53:28.041456 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:28.041438 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:53:30.691994 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.691962 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5"] Jul 08 04:53:30.695399 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.695376 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.698890 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.698871 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-76a8b4\"/\"openshift-service-ca.crt\"" Jul 08 04:53:30.699926 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.699911 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-76a8b4\"/\"test-model-76a8b4-kserve-self-signed-certs\"" Jul 08 04:53:30.700027 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.699978 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-76a8b4\"/\"kube-root-ca.crt\"" Jul 08 04:53:30.700027 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.699980 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-76a8b4\"/\"default-dockercfg-9kzfw\"" Jul 08 04:53:30.708174 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.708155 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5"] Jul 08 04:53:30.775282 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.775245 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-dshm\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.775282 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.775279 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-model-cache\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.775497 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.775323 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-tmp-dir\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.775497 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.775360 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/732db2cd-e56b-4142-87c0-11293c0faa19-tls-certs\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.775497 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.775395 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxbcq\" (UniqueName: \"kubernetes.io/projected/732db2cd-e56b-4142-87c0-11293c0faa19-kube-api-access-qxbcq\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.775497 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.775416 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-home\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.876291 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.876255 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-dshm\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.876291 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.876288 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-model-cache\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.876508 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.876418 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-tmp-dir\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.876508 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.876457 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/732db2cd-e56b-4142-87c0-11293c0faa19-tls-certs\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.876625 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.876515 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxbcq\" (UniqueName: \"kubernetes.io/projected/732db2cd-e56b-4142-87c0-11293c0faa19-kube-api-access-qxbcq\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.876625 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.876558 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-home\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.876804 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.876786 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-model-cache\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.876863 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.876807 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-tmp-dir\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.876863 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.876854 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-home\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.878606 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.878585 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-dshm\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.878845 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.878827 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/732db2cd-e56b-4142-87c0-11293c0faa19-tls-certs\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:30.885541 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:30.885521 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxbcq\" (UniqueName: \"kubernetes.io/projected/732db2cd-e56b-4142-87c0-11293c0faa19-kube-api-access-qxbcq\") pod \"test-model-76a8b4-kserve-7d67d95997-5rqb5\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:31.004378 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:31.004268 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:31.131543 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:31.131514 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5"] Jul 08 04:53:31.134656 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:53:31.134614 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod732db2cd_e56b_4142_87c0_11293c0faa19.slice/crio-f835d56df721e28d2a6ea55576496545ed96973a1c2b57087938a59be947235b WatchSource:0}: Error finding container f835d56df721e28d2a6ea55576496545ed96973a1c2b57087938a59be947235b: Status 404 returned error can't find the container with id f835d56df721e28d2a6ea55576496545ed96973a1c2b57087938a59be947235b Jul 08 04:53:31.683110 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:31.683068 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" event={"ID":"732db2cd-e56b-4142-87c0-11293c0faa19","Type":"ContainerStarted","Data":"73ffd75b7b980f75366bce7773060d15ef2c3b81a5b198b0c6b1ed6e3a44e74f"} Jul 08 04:53:31.683110 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:31.683110 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" event={"ID":"732db2cd-e56b-4142-87c0-11293c0faa19","Type":"ContainerStarted","Data":"f835d56df721e28d2a6ea55576496545ed96973a1c2b57087938a59be947235b"} Jul 08 04:53:31.699647 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:31.699594 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" podStartSLOduration=1.699578166 podStartE2EDuration="1.699578166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:53:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:53:31.698482057 +0000 UTC m=+2104.135257035" watchObservedRunningTime="2026-07-08 04:53:31.699578166 +0000 UTC m=+2104.136353141" Jul 08 04:53:37.026597 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:37.026566 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:53:41.004635 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:41.004602 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:41.005027 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:41.004655 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:41.016687 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:41.016662 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:53:41.734924 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:53:41.734895 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:54:00.467571 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:54:00.467533 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:54:25.932853 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:54:25.932820 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:54:49.638508 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:54:49.638471 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:55:19.928748 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:19.928712 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:55:38.912109 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:38.912072 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj"] Jul 08 04:55:38.915540 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:38.915518 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:38.919525 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:38.919500 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-reuse-a4325b\"/\"openshift-service-ca.crt\"" Jul 08 04:55:38.919652 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:38.919542 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-reuse-a4325b\"/\"kube-root-ca.crt\"" Jul 08 04:55:38.919729 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:38.919646 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-reuse-a4325b\"/\"default-dockercfg-g45hx\"" Jul 08 04:55:38.919729 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:38.919664 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-reuse-a4325b\"/\"test-model-a4325b-kserve-self-signed-certs\"" Jul 08 04:55:38.923303 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:38.923273 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj"] Jul 08 04:55:39.091772 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.091736 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-tmp-dir\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.091772 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.091770 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-model-cache\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.091980 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.091790 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-home\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.091980 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.091808 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s9hh\" (UniqueName: \"kubernetes.io/projected/d30a727f-59eb-4593-8146-a284bcfdbd1b-kube-api-access-9s9hh\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.091980 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.091873 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-dshm\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.091980 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.091898 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d30a727f-59eb-4593-8146-a284bcfdbd1b-tls-certs\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.193244 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.193136 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-tmp-dir\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.193244 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.193229 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-model-cache\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.193497 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.193254 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-home\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.193497 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.193272 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s9hh\" (UniqueName: \"kubernetes.io/projected/d30a727f-59eb-4593-8146-a284bcfdbd1b-kube-api-access-9s9hh\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.193497 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.193442 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-dshm\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.193497 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.193475 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d30a727f-59eb-4593-8146-a284bcfdbd1b-tls-certs\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.193719 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.193551 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-tmp-dir\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.193719 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.193647 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-model-cache\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.193989 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.193964 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-home\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.195693 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.195668 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-dshm\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.195879 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.195863 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d30a727f-59eb-4593-8146-a284bcfdbd1b-tls-certs\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.202005 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.201987 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s9hh\" (UniqueName: \"kubernetes.io/projected/d30a727f-59eb-4593-8146-a284bcfdbd1b-kube-api-access-9s9hh\") pod \"test-model-a4325b-kserve-cdcdb495f-lxdxj\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.226239 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.226213 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:39.356979 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:39.356956 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj"] Jul 08 04:55:39.359242 ip-10-0-133-10 kubenswrapper[2583]: W0708 04:55:39.359216 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd30a727f_59eb_4593_8146_a284bcfdbd1b.slice/crio-3368e274cea22529186c0bdc5fbc0709f24275829e715b6c1c13276c199810cd WatchSource:0}: Error finding container 3368e274cea22529186c0bdc5fbc0709f24275829e715b6c1c13276c199810cd: Status 404 returned error can't find the container with id 3368e274cea22529186c0bdc5fbc0709f24275829e715b6c1c13276c199810cd Jul 08 04:55:40.238282 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:40.238248 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" event={"ID":"d30a727f-59eb-4593-8146-a284bcfdbd1b","Type":"ContainerStarted","Data":"df17a1f92e01d18d3939739607f05c935858f0879ac30f704878f8d6f2231dbe"} Jul 08 04:55:40.238715 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:40.238294 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" event={"ID":"d30a727f-59eb-4593-8146-a284bcfdbd1b","Type":"ContainerStarted","Data":"3368e274cea22529186c0bdc5fbc0709f24275829e715b6c1c13276c199810cd"} Jul 08 04:55:40.259702 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:40.259657 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" podStartSLOduration=2.259640699 podStartE2EDuration="2.259640699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:55:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:55:40.258081427 +0000 UTC m=+2232.694856427" watchObservedRunningTime="2026-07-08 04:55:40.259640699 +0000 UTC m=+2232.696415678" Jul 08 04:55:44.437415 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:44.437379 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:55:49.226378 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:49.226327 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:49.226755 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:49.226394 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:49.239107 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:49.239080 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:55:49.289897 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:55:49.289863 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:56:22.088941 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:22.088911 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj"] Jul 08 04:56:22.927438 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:22.927403 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:56:37.108193 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.108144 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" podUID="d30a727f-59eb-4593-8146-a284bcfdbd1b" containerName="main" containerID="cri-o://df17a1f92e01d18d3939739607f05c935858f0879ac30f704878f8d6f2231dbe" gracePeriod=45 Jul 08 04:56:37.356677 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.356652 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:56:37.407512 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.407485 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d30a727f-59eb-4593-8146-a284bcfdbd1b-tls-certs\") pod \"d30a727f-59eb-4593-8146-a284bcfdbd1b\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " Jul 08 04:56:37.407649 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.407545 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-home\") pod \"d30a727f-59eb-4593-8146-a284bcfdbd1b\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " Jul 08 04:56:37.407649 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.407584 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-dshm\") pod \"d30a727f-59eb-4593-8146-a284bcfdbd1b\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " Jul 08 04:56:37.407765 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.407653 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9s9hh\" (UniqueName: \"kubernetes.io/projected/d30a727f-59eb-4593-8146-a284bcfdbd1b-kube-api-access-9s9hh\") pod \"d30a727f-59eb-4593-8146-a284bcfdbd1b\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " Jul 08 04:56:37.407765 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.407703 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-tmp-dir\") pod \"d30a727f-59eb-4593-8146-a284bcfdbd1b\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " Jul 08 04:56:37.407765 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.407728 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-model-cache\") pod \"d30a727f-59eb-4593-8146-a284bcfdbd1b\" (UID: \"d30a727f-59eb-4593-8146-a284bcfdbd1b\") " Jul 08 04:56:37.407915 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.407795 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-home" (OuterVolumeSpecName: "home") pod "d30a727f-59eb-4593-8146-a284bcfdbd1b" (UID: "d30a727f-59eb-4593-8146-a284bcfdbd1b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:56:37.408034 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.408012 2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-home\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:56:37.408118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.408037 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-model-cache" (OuterVolumeSpecName: "model-cache") pod "d30a727f-59eb-4593-8146-a284bcfdbd1b" (UID: "d30a727f-59eb-4593-8146-a284bcfdbd1b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:56:37.408118 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.408055 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d30a727f-59eb-4593-8146-a284bcfdbd1b" (UID: "d30a727f-59eb-4593-8146-a284bcfdbd1b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:56:37.409751 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.409726 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d30a727f-59eb-4593-8146-a284bcfdbd1b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d30a727f-59eb-4593-8146-a284bcfdbd1b" (UID: "d30a727f-59eb-4593-8146-a284bcfdbd1b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:56:37.409882 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.409792 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-dshm" (OuterVolumeSpecName: "dshm") pod "d30a727f-59eb-4593-8146-a284bcfdbd1b" (UID: "d30a727f-59eb-4593-8146-a284bcfdbd1b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:56:37.409882 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.409833 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d30a727f-59eb-4593-8146-a284bcfdbd1b-kube-api-access-9s9hh" (OuterVolumeSpecName: "kube-api-access-9s9hh") pod "d30a727f-59eb-4593-8146-a284bcfdbd1b" (UID: "d30a727f-59eb-4593-8146-a284bcfdbd1b"). InnerVolumeSpecName "kube-api-access-9s9hh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:56:37.491566 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.491534 2583 generic.go:356] "Generic (PLEG): container finished" podID="d30a727f-59eb-4593-8146-a284bcfdbd1b" containerID="df17a1f92e01d18d3939739607f05c935858f0879ac30f704878f8d6f2231dbe" exitCode=0 Jul 08 04:56:37.491701 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.491575 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" Jul 08 04:56:37.491701 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.491625 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" event={"ID":"d30a727f-59eb-4593-8146-a284bcfdbd1b","Type":"ContainerDied","Data":"df17a1f92e01d18d3939739607f05c935858f0879ac30f704878f8d6f2231dbe"} Jul 08 04:56:37.491701 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.491666 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj" event={"ID":"d30a727f-59eb-4593-8146-a284bcfdbd1b","Type":"ContainerDied","Data":"3368e274cea22529186c0bdc5fbc0709f24275829e715b6c1c13276c199810cd"} Jul 08 04:56:37.491701 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.491685 2583 scope.go:117] "RemoveContainer" containerID="df17a1f92e01d18d3939739607f05c935858f0879ac30f704878f8d6f2231dbe" Jul 08 04:56:37.500177 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.500162 2583 scope.go:117] "RemoveContainer" containerID="df17a1f92e01d18d3939739607f05c935858f0879ac30f704878f8d6f2231dbe" Jul 08 04:56:37.500418 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:56:37.500390 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df17a1f92e01d18d3939739607f05c935858f0879ac30f704878f8d6f2231dbe\": container with ID starting with df17a1f92e01d18d3939739607f05c935858f0879ac30f704878f8d6f2231dbe not found: ID does not exist" containerID="df17a1f92e01d18d3939739607f05c935858f0879ac30f704878f8d6f2231dbe" Jul 08 04:56:37.500515 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.500426 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df17a1f92e01d18d3939739607f05c935858f0879ac30f704878f8d6f2231dbe"} err="failed to get container status \"df17a1f92e01d18d3939739607f05c935858f0879ac30f704878f8d6f2231dbe\": rpc error: code = NotFound desc = could not find container \"df17a1f92e01d18d3939739607f05c935858f0879ac30f704878f8d6f2231dbe\": container with ID starting with df17a1f92e01d18d3939739607f05c935858f0879ac30f704878f8d6f2231dbe not found: ID does not exist" Jul 08 04:56:37.508712 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.508693 2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-model-cache\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:56:37.508813 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.508714 2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d30a727f-59eb-4593-8146-a284bcfdbd1b-tls-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:56:37.508813 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.508727 2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-dshm\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:56:37.508813 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.508738 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9s9hh\" (UniqueName: \"kubernetes.io/projected/d30a727f-59eb-4593-8146-a284bcfdbd1b-kube-api-access-9s9hh\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:56:37.508813 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.508775 2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d30a727f-59eb-4593-8146-a284bcfdbd1b-tmp-dir\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:56:37.511338 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.511304 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj"] Jul 08 04:56:37.517334 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:37.517314 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-reuse-a4325b/test-model-a4325b-kserve-cdcdb495f-lxdxj"] Jul 08 04:56:38.073754 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:56:38.073723 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d30a727f-59eb-4593-8146-a284bcfdbd1b" path="/var/lib/kubelet/pods/d30a727f-59eb-4593-8146-a284bcfdbd1b/volumes" Jul 08 04:57:02.941250 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:02.941218 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:57:06.968912 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:06.968872 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727"] Jul 08 04:57:07.996767 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:07.996733 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5"] Jul 08 04:57:21.988779 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:21.988717 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" podUID="372e6930-a79a-40c0-b412-148d013047d8" containerName="main" containerID="cri-o://a66b173971077ebe675808ebbfb494b63d41f9754560cb4aeee8d91f6f8829b1" gracePeriod=45 Jul 08 04:57:22.032621 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.032595 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:57:22.232515 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.232486 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:57:22.288076 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.288007 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4plg\" (UniqueName: \"kubernetes.io/projected/372e6930-a79a-40c0-b412-148d013047d8-kube-api-access-l4plg\") pod \"372e6930-a79a-40c0-b412-148d013047d8\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " Jul 08 04:57:22.288076 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.288059 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/372e6930-a79a-40c0-b412-148d013047d8-tls-certs\") pod \"372e6930-a79a-40c0-b412-148d013047d8\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " Jul 08 04:57:22.288261 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.288103 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-tmp-dir\") pod \"372e6930-a79a-40c0-b412-148d013047d8\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " Jul 08 04:57:22.288261 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.288133 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-dshm\") pod \"372e6930-a79a-40c0-b412-148d013047d8\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " Jul 08 04:57:22.288261 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.288206 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-model-cache\") pod \"372e6930-a79a-40c0-b412-148d013047d8\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " Jul 08 04:57:22.288439 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.288296 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-home\") pod \"372e6930-a79a-40c0-b412-148d013047d8\" (UID: \"372e6930-a79a-40c0-b412-148d013047d8\") " Jul 08 04:57:22.288587 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.288453 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "372e6930-a79a-40c0-b412-148d013047d8" (UID: "372e6930-a79a-40c0-b412-148d013047d8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:57:22.288687 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.288635 2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-tmp-dir\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:57:22.288687 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.288606 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-model-cache" (OuterVolumeSpecName: "model-cache") pod "372e6930-a79a-40c0-b412-148d013047d8" (UID: "372e6930-a79a-40c0-b412-148d013047d8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:57:22.288687 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.288669 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-home" (OuterVolumeSpecName: "home") pod "372e6930-a79a-40c0-b412-148d013047d8" (UID: "372e6930-a79a-40c0-b412-148d013047d8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:57:22.290393 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.290367 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/372e6930-a79a-40c0-b412-148d013047d8-kube-api-access-l4plg" (OuterVolumeSpecName: "kube-api-access-l4plg") pod "372e6930-a79a-40c0-b412-148d013047d8" (UID: "372e6930-a79a-40c0-b412-148d013047d8"). InnerVolumeSpecName "kube-api-access-l4plg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:57:22.290393 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.290388 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-dshm" (OuterVolumeSpecName: "dshm") pod "372e6930-a79a-40c0-b412-148d013047d8" (UID: "372e6930-a79a-40c0-b412-148d013047d8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:57:22.290506 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.290401 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/372e6930-a79a-40c0-b412-148d013047d8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "372e6930-a79a-40c0-b412-148d013047d8" (UID: "372e6930-a79a-40c0-b412-148d013047d8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:57:22.389136 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.389108 2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/372e6930-a79a-40c0-b412-148d013047d8-tls-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:57:22.389136 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.389133 2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-dshm\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:57:22.389136 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.389142 2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-model-cache\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:57:22.389365 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.389149 2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/372e6930-a79a-40c0-b412-148d013047d8-home\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:57:22.389365 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.389157 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4plg\" (UniqueName: \"kubernetes.io/projected/372e6930-a79a-40c0-b412-148d013047d8-kube-api-access-l4plg\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:57:22.680297 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.680263 2583 generic.go:356] "Generic (PLEG): container finished" podID="372e6930-a79a-40c0-b412-148d013047d8" containerID="a66b173971077ebe675808ebbfb494b63d41f9754560cb4aeee8d91f6f8829b1" exitCode=0 Jul 08 04:57:22.680297 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.680300 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" Jul 08 04:57:22.680513 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.680367 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" event={"ID":"372e6930-a79a-40c0-b412-148d013047d8","Type":"ContainerDied","Data":"a66b173971077ebe675808ebbfb494b63d41f9754560cb4aeee8d91f6f8829b1"} Jul 08 04:57:22.680513 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.680420 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727" event={"ID":"372e6930-a79a-40c0-b412-148d013047d8","Type":"ContainerDied","Data":"dc5c50afd6b48c54b62ce262943eb68ded5e587c4257ae110df9101be5639820"} Jul 08 04:57:22.680513 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.680437 2583 scope.go:117] "RemoveContainer" containerID="a66b173971077ebe675808ebbfb494b63d41f9754560cb4aeee8d91f6f8829b1" Jul 08 04:57:22.689137 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.689121 2583 scope.go:117] "RemoveContainer" containerID="a66b173971077ebe675808ebbfb494b63d41f9754560cb4aeee8d91f6f8829b1" Jul 08 04:57:22.689397 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:57:22.689375 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a66b173971077ebe675808ebbfb494b63d41f9754560cb4aeee8d91f6f8829b1\": container with ID starting with a66b173971077ebe675808ebbfb494b63d41f9754560cb4aeee8d91f6f8829b1 not found: ID does not exist" containerID="a66b173971077ebe675808ebbfb494b63d41f9754560cb4aeee8d91f6f8829b1" Jul 08 04:57:22.689472 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.689405 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a66b173971077ebe675808ebbfb494b63d41f9754560cb4aeee8d91f6f8829b1"} err="failed to get container status \"a66b173971077ebe675808ebbfb494b63d41f9754560cb4aeee8d91f6f8829b1\": rpc error: code = NotFound desc = could not find container \"a66b173971077ebe675808ebbfb494b63d41f9754560cb4aeee8d91f6f8829b1\": container with ID starting with a66b173971077ebe675808ebbfb494b63d41f9754560cb4aeee8d91f6f8829b1 not found: ID does not exist" Jul 08 04:57:22.700056 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.700031 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727"] Jul 08 04:57:22.704772 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:22.704753 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-f87c81/test-model-f87c81-kserve-5bb8587ff-fr727"] Jul 08 04:57:23.016769 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.016675 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" podUID="732db2cd-e56b-4142-87c0-11293c0faa19" containerName="main" containerID="cri-o://73ffd75b7b980f75366bce7773060d15ef2c3b81a5b198b0c6b1ed6e3a44e74f" gracePeriod=45 Jul 08 04:57:23.260047 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.260023 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:57:23.302091 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.302024 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/732db2cd-e56b-4142-87c0-11293c0faa19-tls-certs\") pod \"732db2cd-e56b-4142-87c0-11293c0faa19\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " Jul 08 04:57:23.302091 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.302079 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-model-cache\") pod \"732db2cd-e56b-4142-87c0-11293c0faa19\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " Jul 08 04:57:23.302294 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.302125 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-dshm\") pod \"732db2cd-e56b-4142-87c0-11293c0faa19\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " Jul 08 04:57:23.302294 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.302143 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxbcq\" (UniqueName: \"kubernetes.io/projected/732db2cd-e56b-4142-87c0-11293c0faa19-kube-api-access-qxbcq\") pod \"732db2cd-e56b-4142-87c0-11293c0faa19\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " Jul 08 04:57:23.302294 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.302161 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-tmp-dir\") pod \"732db2cd-e56b-4142-87c0-11293c0faa19\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " Jul 08 04:57:23.302294 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.302179 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-home\") pod \"732db2cd-e56b-4142-87c0-11293c0faa19\" (UID: \"732db2cd-e56b-4142-87c0-11293c0faa19\") " Jul 08 04:57:23.302523 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.302406 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-model-cache" (OuterVolumeSpecName: "model-cache") pod "732db2cd-e56b-4142-87c0-11293c0faa19" (UID: "732db2cd-e56b-4142-87c0-11293c0faa19"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:57:23.302523 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.302436 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "732db2cd-e56b-4142-87c0-11293c0faa19" (UID: "732db2cd-e56b-4142-87c0-11293c0faa19"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:57:23.302617 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.302531 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-home" (OuterVolumeSpecName: "home") pod "732db2cd-e56b-4142-87c0-11293c0faa19" (UID: "732db2cd-e56b-4142-87c0-11293c0faa19"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:57:23.304360 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.304323 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/732db2cd-e56b-4142-87c0-11293c0faa19-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "732db2cd-e56b-4142-87c0-11293c0faa19" (UID: "732db2cd-e56b-4142-87c0-11293c0faa19"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 04:57:23.304478 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.304386 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-dshm" (OuterVolumeSpecName: "dshm") pod "732db2cd-e56b-4142-87c0-11293c0faa19" (UID: "732db2cd-e56b-4142-87c0-11293c0faa19"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 04:57:23.304478 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.304405 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/732db2cd-e56b-4142-87c0-11293c0faa19-kube-api-access-qxbcq" (OuterVolumeSpecName: "kube-api-access-qxbcq") pod "732db2cd-e56b-4142-87c0-11293c0faa19" (UID: "732db2cd-e56b-4142-87c0-11293c0faa19"). InnerVolumeSpecName "kube-api-access-qxbcq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 04:57:23.403486 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.403459 2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/732db2cd-e56b-4142-87c0-11293c0faa19-tls-certs\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:57:23.403486 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.403481 2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-model-cache\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:57:23.403486 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.403490 2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-dshm\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:57:23.403684 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.403498 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxbcq\" (UniqueName: \"kubernetes.io/projected/732db2cd-e56b-4142-87c0-11293c0faa19-kube-api-access-qxbcq\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:57:23.403684 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.403507 2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-tmp-dir\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:57:23.403684 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.403514 2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/732db2cd-e56b-4142-87c0-11293c0faa19-home\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 04:57:23.687807 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.687771 2583 generic.go:356] "Generic (PLEG): container finished" podID="732db2cd-e56b-4142-87c0-11293c0faa19" containerID="73ffd75b7b980f75366bce7773060d15ef2c3b81a5b198b0c6b1ed6e3a44e74f" exitCode=0 Jul 08 04:57:23.687967 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.687815 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" Jul 08 04:57:23.687967 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.687831 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" event={"ID":"732db2cd-e56b-4142-87c0-11293c0faa19","Type":"ContainerDied","Data":"73ffd75b7b980f75366bce7773060d15ef2c3b81a5b198b0c6b1ed6e3a44e74f"} Jul 08 04:57:23.687967 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.687869 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5" event={"ID":"732db2cd-e56b-4142-87c0-11293c0faa19","Type":"ContainerDied","Data":"f835d56df721e28d2a6ea55576496545ed96973a1c2b57087938a59be947235b"} Jul 08 04:57:23.687967 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.687890 2583 scope.go:117] "RemoveContainer" containerID="73ffd75b7b980f75366bce7773060d15ef2c3b81a5b198b0c6b1ed6e3a44e74f" Jul 08 04:57:23.696660 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.696641 2583 scope.go:117] "RemoveContainer" containerID="73ffd75b7b980f75366bce7773060d15ef2c3b81a5b198b0c6b1ed6e3a44e74f" Jul 08 04:57:23.696934 ip-10-0-133-10 kubenswrapper[2583]: E0708 04:57:23.696916 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73ffd75b7b980f75366bce7773060d15ef2c3b81a5b198b0c6b1ed6e3a44e74f\": container with ID starting with 73ffd75b7b980f75366bce7773060d15ef2c3b81a5b198b0c6b1ed6e3a44e74f not found: ID does not exist" containerID="73ffd75b7b980f75366bce7773060d15ef2c3b81a5b198b0c6b1ed6e3a44e74f" Jul 08 04:57:23.697014 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.696946 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73ffd75b7b980f75366bce7773060d15ef2c3b81a5b198b0c6b1ed6e3a44e74f"} err="failed to get container status \"73ffd75b7b980f75366bce7773060d15ef2c3b81a5b198b0c6b1ed6e3a44e74f\": rpc error: code = NotFound desc = could not find container \"73ffd75b7b980f75366bce7773060d15ef2c3b81a5b198b0c6b1ed6e3a44e74f\": container with ID starting with 73ffd75b7b980f75366bce7773060d15ef2c3b81a5b198b0c6b1ed6e3a44e74f not found: ID does not exist" Jul 08 04:57:23.707611 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.707589 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5"] Jul 08 04:57:23.709933 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:23.709913 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-76a8b4/test-model-76a8b4-kserve-7d67d95997-5rqb5"] Jul 08 04:57:24.074395 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:24.074302 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="372e6930-a79a-40c0-b412-148d013047d8" path="/var/lib/kubelet/pods/372e6930-a79a-40c0-b412-148d013047d8/volumes" Jul 08 04:57:24.074730 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:24.074700 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="732db2cd-e56b-4142-87c0-11293c0faa19" path="/var/lib/kubelet/pods/732db2cd-e56b-4142-87c0-11293c0faa19/volumes" Jul 08 04:57:53.947724 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:57:53.947687 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:58:06.932057 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:58:06.932025 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:58:17.924567 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:58:17.924533 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:58:28.062401 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:58:28.062374 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:58:28.065473 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:58:28.065454 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-133-10.ec2.internal_26ba4c92c69564f7ea9cda441d8bf34b/kube-rbac-proxy-crio/1.log" Jul 08 04:58:34.131314 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:58:34.131278 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:58:46.330367 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:58:46.330322 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 04:58:59.035672 ip-10-0-133-10 kubenswrapper[2583]: I0708 04:58:59.035636 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 05:00:00.140510 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.140476 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299"] Jul 08 05:00:00.140915 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.140777 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d30a727f-59eb-4593-8146-a284bcfdbd1b" containerName="main" Jul 08 05:00:00.140915 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.140787 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d30a727f-59eb-4593-8146-a284bcfdbd1b" containerName="main" Jul 08 05:00:00.140915 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.140800 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="732db2cd-e56b-4142-87c0-11293c0faa19" containerName="main" Jul 08 05:00:00.140915 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.140806 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="732db2cd-e56b-4142-87c0-11293c0faa19" containerName="main" Jul 08 05:00:00.140915 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.140815 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="372e6930-a79a-40c0-b412-148d013047d8" containerName="main" Jul 08 05:00:00.140915 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.140820 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="372e6930-a79a-40c0-b412-148d013047d8" containerName="main" Jul 08 05:00:00.140915 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.140872 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="732db2cd-e56b-4142-87c0-11293c0faa19" containerName="main" Jul 08 05:00:00.140915 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.140879 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d30a727f-59eb-4593-8146-a284bcfdbd1b" containerName="main" Jul 08 05:00:00.140915 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.140886 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="372e6930-a79a-40c0-b412-148d013047d8" containerName="main" Jul 08 05:00:00.143761 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.143740 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299" Jul 08 05:00:00.146320 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.146298 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 05:00:00.146454 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.146437 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 05:00:00.147434 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.147412 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7lh5k\"" Jul 08 05:00:00.152807 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.152772 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299"] Jul 08 05:00:00.256847 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.256816 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcpb5\" (UniqueName: \"kubernetes.io/projected/9ccde567-62dd-4ba1-a65d-75000cc0188d-kube-api-access-qcpb5\") pod \"maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299\" (UID: \"9ccde567-62dd-4ba1-a65d-75000cc0188d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299" Jul 08 05:00:00.357720 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.357684 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcpb5\" (UniqueName: \"kubernetes.io/projected/9ccde567-62dd-4ba1-a65d-75000cc0188d-kube-api-access-qcpb5\") pod \"maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299\" (UID: \"9ccde567-62dd-4ba1-a65d-75000cc0188d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299" Jul 08 05:00:00.365485 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.365459 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcpb5\" (UniqueName: \"kubernetes.io/projected/9ccde567-62dd-4ba1-a65d-75000cc0188d-kube-api-access-qcpb5\") pod \"maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299\" (UID: \"9ccde567-62dd-4ba1-a65d-75000cc0188d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299" Jul 08 05:00:00.455870 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.455829 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299" Jul 08 05:00:00.580579 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.580557 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299"] Jul 08 05:00:00.582559 ip-10-0-133-10 kubenswrapper[2583]: W0708 05:00:00.582529 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ccde567_62dd_4ba1_a65d_75000cc0188d.slice/crio-fd0202bf1a12981bf16bdb8934c901f5d8b7ddf6663e1c954a94a25f786c51a4 WatchSource:0}: Error finding container fd0202bf1a12981bf16bdb8934c901f5d8b7ddf6663e1c954a94a25f786c51a4: Status 404 returned error can't find the container with id fd0202bf1a12981bf16bdb8934c901f5d8b7ddf6663e1c954a94a25f786c51a4 Jul 08 05:00:00.584454 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:00.584439 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 05:00:01.364638 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:01.364606 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299" event={"ID":"9ccde567-62dd-4ba1-a65d-75000cc0188d","Type":"ContainerStarted","Data":"fd0202bf1a12981bf16bdb8934c901f5d8b7ddf6663e1c954a94a25f786c51a4"} Jul 08 05:00:06.388748 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:06.388710 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299" event={"ID":"9ccde567-62dd-4ba1-a65d-75000cc0188d","Type":"ContainerStarted","Data":"f52f8195efa8943fbac9b5a00e963f3a22e93905fb0c35876441ffcfb9114363"} Jul 08 05:00:06.405734 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:00:06.405682 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299" podStartSLOduration=1.48264097 podStartE2EDuration="6.405663168s" totalImagesPullingTime="4.923022198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 05:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 05:00:06.404511021 +0000 UTC m=+2498.841286001" watchObservedRunningTime="2026-07-08 05:00:06.405663168 +0000 UTC m=+2498.842438149" Jul 08 05:01:00.014167 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:01:00.014129 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299"] Jul 08 05:01:00.014740 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:01:00.014362 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299" podUID="9ccde567-62dd-4ba1-a65d-75000cc0188d" containerName="cleanup" containerID="cri-o://f52f8195efa8943fbac9b5a00e963f3a22e93905fb0c35876441ffcfb9114363" gracePeriod=30 Jul 08 05:01:30.197476 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:01:30.197448 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299" Jul 08 05:01:30.318181 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:01:30.318091 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcpb5\" (UniqueName: \"kubernetes.io/projected/9ccde567-62dd-4ba1-a65d-75000cc0188d-kube-api-access-qcpb5\") pod \"9ccde567-62dd-4ba1-a65d-75000cc0188d\" (UID: \"9ccde567-62dd-4ba1-a65d-75000cc0188d\") " Jul 08 05:01:30.320233 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:01:30.320208 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ccde567-62dd-4ba1-a65d-75000cc0188d-kube-api-access-qcpb5" (OuterVolumeSpecName: "kube-api-access-qcpb5") pod "9ccde567-62dd-4ba1-a65d-75000cc0188d" (UID: "9ccde567-62dd-4ba1-a65d-75000cc0188d"). InnerVolumeSpecName "kube-api-access-qcpb5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 05:01:30.419468 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:01:30.419438 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcpb5\" (UniqueName: \"kubernetes.io/projected/9ccde567-62dd-4ba1-a65d-75000cc0188d-kube-api-access-qcpb5\") on node \"ip-10-0-133-10.ec2.internal\" DevicePath \"\"" Jul 08 05:01:30.742254 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:01:30.742217 2583 generic.go:356] "Generic (PLEG): container finished" podID="9ccde567-62dd-4ba1-a65d-75000cc0188d" containerID="f52f8195efa8943fbac9b5a00e963f3a22e93905fb0c35876441ffcfb9114363" exitCode=137 Jul 08 05:01:30.742432 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:01:30.742266 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299" Jul 08 05:01:30.742432 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:01:30.742301 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299" event={"ID":"9ccde567-62dd-4ba1-a65d-75000cc0188d","Type":"ContainerDied","Data":"f52f8195efa8943fbac9b5a00e963f3a22e93905fb0c35876441ffcfb9114363"} Jul 08 05:01:30.742432 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:01:30.742369 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299" event={"ID":"9ccde567-62dd-4ba1-a65d-75000cc0188d","Type":"ContainerDied","Data":"fd0202bf1a12981bf16bdb8934c901f5d8b7ddf6663e1c954a94a25f786c51a4"} Jul 08 05:01:30.742432 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:01:30.742401 2583 scope.go:117] "RemoveContainer" containerID="f52f8195efa8943fbac9b5a00e963f3a22e93905fb0c35876441ffcfb9114363" Jul 08 05:01:30.750528 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:01:30.750505 2583 scope.go:117] "RemoveContainer" containerID="f52f8195efa8943fbac9b5a00e963f3a22e93905fb0c35876441ffcfb9114363" Jul 08 05:01:30.750798 ip-10-0-133-10 kubenswrapper[2583]: E0708 05:01:30.750776 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f52f8195efa8943fbac9b5a00e963f3a22e93905fb0c35876441ffcfb9114363\": container with ID starting with f52f8195efa8943fbac9b5a00e963f3a22e93905fb0c35876441ffcfb9114363 not found: ID does not exist" containerID="f52f8195efa8943fbac9b5a00e963f3a22e93905fb0c35876441ffcfb9114363" Jul 08 05:01:30.750882 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:01:30.750805 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f52f8195efa8943fbac9b5a00e963f3a22e93905fb0c35876441ffcfb9114363"} err="failed to get container status \"f52f8195efa8943fbac9b5a00e963f3a22e93905fb0c35876441ffcfb9114363\": rpc error: code = NotFound desc = could not find container \"f52f8195efa8943fbac9b5a00e963f3a22e93905fb0c35876441ffcfb9114363\": container with ID starting with f52f8195efa8943fbac9b5a00e963f3a22e93905fb0c35876441ffcfb9114363 not found: ID does not exist" Jul 08 05:01:30.763104 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:01:30.763080 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299"] Jul 08 05:01:30.768089 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:01:30.768069 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bd94ec-29724780-hj299"] Jul 08 05:01:32.073729 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:01:32.073697 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ccde567-62dd-4ba1-a65d-75000cc0188d" path="/var/lib/kubelet/pods/9ccde567-62dd-4ba1-a65d-75000cc0188d/volumes" Jul 08 05:02:00.219734 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:00.219702 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 05:02:07.928493 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:07.928465 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4s4z"] Jul 08 05:02:10.030546 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:10.030518 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f868c74df-9d9js_d98f2f76-390d-425e-b338-2cc0decfa194/authorino/0.log" Jul 08 05:02:14.450915 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:14.450882 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7h8bq_5a0cf4c9-10e4-4904-b1aa-3dba4d42e707/manager/0.log" Jul 08 05:02:14.809048 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:14.808983 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-h4qqn_0d62104b-a8cf-46d9-bafa-4f0580fe4575/server/0.log" Jul 08 05:02:14.918450 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:14.918427 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-c8bhm_313b79f2-b436-4920-92c7-04fdddf2e58c/manager/0.log" Jul 08 05:02:15.153234 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:15.153208 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f76f945bd-csl6v_8ee834ca-5cac-4c2b-a1b2-2a9764e8489a/manager/0.log" Jul 08 05:02:15.166207 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:15.166183 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f76f945bd-csl6v_8ee834ca-5cac-4c2b-a1b2-2a9764e8489a/copy-manifests/0.log" Jul 08 05:02:16.563674 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:16.563624 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f868c74df-9d9js_d98f2f76-390d-425e-b338-2cc0decfa194/authorino/0.log" Jul 08 05:02:16.906610 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:16.906580 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-prbsm_747a0f10-9ba7-4402-b601-dbcf0d2ecf66/kuadrant-console-plugin/0.log" Jul 08 05:02:17.255191 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:17.255114 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-v4s4z_98a7af65-5b90-4d3a-b8b4-93a03b8187b2/limitador/0.log" Jul 08 05:02:17.934249 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:17.934196 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-668854d458-4ctbp_d847cbd1-645f-4266-8ad4-05ae2ac443e1/kube-auth-proxy/0.log" Jul 08 05:02:19.167530 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:19.167504 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbvjx2_6e2b3851-0b99-4e9a-8bdc-446f59239f35/main/0.log" Jul 08 05:02:19.389977 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:19.389950 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-crndh_8dd00104-f6d8-4400-9658-b85abef5b854/main/0.log" Jul 08 05:02:26.450268 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:26.450214 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-58wwx_471fcdae-c4d6-4999-b260-0f0719e99682/global-pull-secret-syncer/0.log" Jul 08 05:02:26.721193 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:26.721102 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-m5qfn_986153ab-266b-49f8-846e-cdde4d6b90af/konnectivity-agent/0.log" Jul 08 05:02:26.742728 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:26.742703 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-10.ec2.internal_114dd0440d9cd8b63ab11e6e15492e28/haproxy/0.log" Jul 08 05:02:32.050385 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:32.050270 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f868c74df-9d9js_d98f2f76-390d-425e-b338-2cc0decfa194/authorino/0.log" Jul 08 05:02:32.128085 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:32.128055 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-prbsm_747a0f10-9ba7-4402-b601-dbcf0d2ecf66/kuadrant-console-plugin/0.log" Jul 08 05:02:32.350957 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:32.350883 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-v4s4z_98a7af65-5b90-4d3a-b8b4-93a03b8187b2/limitador/0.log" Jul 08 05:02:34.513252 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:34.513182 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mfkzr_504a69c8-25ec-485e-ae7c-d75a74f1620d/node-exporter/0.log" Jul 08 05:02:34.534472 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:34.534446 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mfkzr_504a69c8-25ec-485e-ae7c-d75a74f1620d/kube-rbac-proxy/0.log" Jul 08 05:02:34.555654 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:34.555632 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mfkzr_504a69c8-25ec-485e-ae7c-d75a74f1620d/init-textfile/0.log" Jul 08 05:02:34.758478 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:34.758451 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bbb8689d-8508-4493-9021-34220781027e/prometheus/0.log" Jul 08 05:02:34.778008 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:34.777948 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bbb8689d-8508-4493-9021-34220781027e/config-reloader/0.log" Jul 08 05:02:34.797846 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:34.797825 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bbb8689d-8508-4493-9021-34220781027e/thanos-sidecar/0.log" Jul 08 05:02:34.818267 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:34.818249 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bbb8689d-8508-4493-9021-34220781027e/kube-rbac-proxy-web/0.log" Jul 08 05:02:34.841843 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:34.841820 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bbb8689d-8508-4493-9021-34220781027e/kube-rbac-proxy/0.log" Jul 08 05:02:34.870519 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:34.870499 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bbb8689d-8508-4493-9021-34220781027e/kube-rbac-proxy-thanos/0.log" Jul 08 05:02:34.899359 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:34.899328 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bbb8689d-8508-4493-9021-34220781027e/init-config-reloader/0.log" Jul 08 05:02:34.994695 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:34.994666 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c79c98b48-lrbzw_5c80577e-8424-4730-9d52-a621396c324c/telemeter-client/0.log" Jul 08 05:02:35.015318 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:35.015289 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c79c98b48-lrbzw_5c80577e-8424-4730-9d52-a621396c324c/reload/0.log" Jul 08 05:02:35.036060 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:35.036000 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c79c98b48-lrbzw_5c80577e-8424-4730-9d52-a621396c324c/kube-rbac-proxy/0.log" Jul 08 05:02:36.551991 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.551961 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk"] Jul 08 05:02:36.552379 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.552276 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ccde567-62dd-4ba1-a65d-75000cc0188d" containerName="cleanup" Jul 08 05:02:36.552379 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.552287 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ccde567-62dd-4ba1-a65d-75000cc0188d" containerName="cleanup" Jul 08 05:02:36.552481 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.552392 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ccde567-62dd-4ba1-a65d-75000cc0188d" containerName="cleanup" Jul 08 05:02:36.555129 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.555113 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.557647 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.557621 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vmbkg\"/\"openshift-service-ca.crt\"" Jul 08 05:02:36.557758 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.557668 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vmbkg\"/\"kube-root-ca.crt\"" Jul 08 05:02:36.557814 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.557757 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vmbkg\"/\"default-dockercfg-gwwrx\"" Jul 08 05:02:36.565222 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.565201 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk"] Jul 08 05:02:36.679284 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.679257 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8gl8\" (UniqueName: \"kubernetes.io/projected/ed8fa76c-1f38-4c05-96de-b6253b6113c0-kube-api-access-s8gl8\") pod \"perf-node-gather-daemonset-q4nvk\" (UID: \"ed8fa76c-1f38-4c05-96de-b6253b6113c0\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.679434 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.679303 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ed8fa76c-1f38-4c05-96de-b6253b6113c0-podres\") pod \"perf-node-gather-daemonset-q4nvk\" (UID: \"ed8fa76c-1f38-4c05-96de-b6253b6113c0\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.679434 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.679359 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ed8fa76c-1f38-4c05-96de-b6253b6113c0-lib-modules\") pod \"perf-node-gather-daemonset-q4nvk\" (UID: \"ed8fa76c-1f38-4c05-96de-b6253b6113c0\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.679434 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.679387 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed8fa76c-1f38-4c05-96de-b6253b6113c0-sys\") pod \"perf-node-gather-daemonset-q4nvk\" (UID: \"ed8fa76c-1f38-4c05-96de-b6253b6113c0\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.679544 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.679437 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ed8fa76c-1f38-4c05-96de-b6253b6113c0-proc\") pod \"perf-node-gather-daemonset-q4nvk\" (UID: \"ed8fa76c-1f38-4c05-96de-b6253b6113c0\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.779801 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.779771 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8gl8\" (UniqueName: \"kubernetes.io/projected/ed8fa76c-1f38-4c05-96de-b6253b6113c0-kube-api-access-s8gl8\") pod \"perf-node-gather-daemonset-q4nvk\" (UID: \"ed8fa76c-1f38-4c05-96de-b6253b6113c0\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.779950 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.779813 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ed8fa76c-1f38-4c05-96de-b6253b6113c0-podres\") pod \"perf-node-gather-daemonset-q4nvk\" (UID: \"ed8fa76c-1f38-4c05-96de-b6253b6113c0\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.779950 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.779836 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ed8fa76c-1f38-4c05-96de-b6253b6113c0-lib-modules\") pod \"perf-node-gather-daemonset-q4nvk\" (UID: \"ed8fa76c-1f38-4c05-96de-b6253b6113c0\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.779950 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.779861 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed8fa76c-1f38-4c05-96de-b6253b6113c0-sys\") pod \"perf-node-gather-daemonset-q4nvk\" (UID: \"ed8fa76c-1f38-4c05-96de-b6253b6113c0\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.779950 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.779894 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ed8fa76c-1f38-4c05-96de-b6253b6113c0-proc\") pod \"perf-node-gather-daemonset-q4nvk\" (UID: \"ed8fa76c-1f38-4c05-96de-b6253b6113c0\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.780095 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.779973 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ed8fa76c-1f38-4c05-96de-b6253b6113c0-proc\") pod \"perf-node-gather-daemonset-q4nvk\" (UID: \"ed8fa76c-1f38-4c05-96de-b6253b6113c0\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.780095 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.779991 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed8fa76c-1f38-4c05-96de-b6253b6113c0-sys\") pod \"perf-node-gather-daemonset-q4nvk\" (UID: \"ed8fa76c-1f38-4c05-96de-b6253b6113c0\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.780095 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.779971 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ed8fa76c-1f38-4c05-96de-b6253b6113c0-podres\") pod \"perf-node-gather-daemonset-q4nvk\" (UID: \"ed8fa76c-1f38-4c05-96de-b6253b6113c0\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.780095 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.780013 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ed8fa76c-1f38-4c05-96de-b6253b6113c0-lib-modules\") pod \"perf-node-gather-daemonset-q4nvk\" (UID: \"ed8fa76c-1f38-4c05-96de-b6253b6113c0\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.787582 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.787561 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8gl8\" (UniqueName: \"kubernetes.io/projected/ed8fa76c-1f38-4c05-96de-b6253b6113c0-kube-api-access-s8gl8\") pod \"perf-node-gather-daemonset-q4nvk\" (UID: \"ed8fa76c-1f38-4c05-96de-b6253b6113c0\") " pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.866759 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.866686 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:36.988926 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:36.988900 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk"] Jul 08 05:02:36.990728 ip-10-0-133-10 kubenswrapper[2583]: W0708 05:02:36.990695 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poded8fa76c_1f38_4c05_96de_b6253b6113c0.slice/crio-d267c1fd7835ea20d820919f189a25015ac03159821df35c6ed5f7e2d82ce148 WatchSource:0}: Error finding container d267c1fd7835ea20d820919f189a25015ac03159821df35c6ed5f7e2d82ce148: Status 404 returned error can't find the container with id d267c1fd7835ea20d820919f189a25015ac03159821df35c6ed5f7e2d82ce148 Jul 08 05:02:37.016048 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:37.016024 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" event={"ID":"ed8fa76c-1f38-4c05-96de-b6253b6113c0","Type":"ContainerStarted","Data":"d267c1fd7835ea20d820919f189a25015ac03159821df35c6ed5f7e2d82ce148"} Jul 08 05:02:38.021876 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:38.021842 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" event={"ID":"ed8fa76c-1f38-4c05-96de-b6253b6113c0","Type":"ContainerStarted","Data":"5825a276c5848ce500622a4e4f82050cc5109bdb7e61445a3437b93db1ab2d2c"} Jul 08 05:02:38.022262 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:38.021968 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:38.037404 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:38.037333 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" podStartSLOduration=2.037319319 podStartE2EDuration="2.037319319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 05:02:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 05:02:38.036939757 +0000 UTC m=+2650.473714736" watchObservedRunningTime="2026-07-08 05:02:38.037319319 +0000 UTC m=+2650.474094298" Jul 08 05:02:39.271849 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:39.271824 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8mn4h_e6ad9554-4baa-4b5c-8e75-b6099a656f0c/dns/0.log" Jul 08 05:02:39.292448 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:39.292430 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8mn4h_e6ad9554-4baa-4b5c-8e75-b6099a656f0c/kube-rbac-proxy/0.log" Jul 08 05:02:39.382024 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:39.382000 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-slm66_03875a1c-3c96-419a-88a5-631347a6cd00/dns-node-resolver/0.log" Jul 08 05:02:39.945664 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:39.945635 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4rws5_5331e3cb-cabb-4cd9-b7ca-5977c7acaab7/node-ca/0.log" Jul 08 05:02:41.113460 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:41.113435 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-668854d458-4ctbp_d847cbd1-645f-4266-8ad4-05ae2ac443e1/kube-auth-proxy/0.log" Jul 08 05:02:41.996990 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:41.996965 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jf6z2_acdc3d88-e479-4cf3-9fe1-31fd5c1b4554/serve-healthcheck-canary/0.log" Jul 08 05:02:42.594702 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:42.594677 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mgllq_558b5c81-e1c2-4914-8395-d1fe61e9b0a4/kube-rbac-proxy/0.log" Jul 08 05:02:42.614933 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:42.614911 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mgllq_558b5c81-e1c2-4914-8395-d1fe61e9b0a4/exporter/0.log" Jul 08 05:02:42.634303 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:42.634271 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mgllq_558b5c81-e1c2-4914-8395-d1fe61e9b0a4/extractor/0.log" Jul 08 05:02:44.035512 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:44.035481 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vmbkg/perf-node-gather-daemonset-q4nvk" Jul 08 05:02:44.771854 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:44.771823 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7h8bq_5a0cf4c9-10e4-4904-b1aa-3dba4d42e707/manager/0.log" Jul 08 05:02:45.206727 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:45.206700 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-h4qqn_0d62104b-a8cf-46d9-bafa-4f0580fe4575/server/0.log" Jul 08 05:02:45.291169 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:45.291145 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-c8bhm_313b79f2-b436-4920-92c7-04fdddf2e58c/manager/0.log" Jul 08 05:02:45.426378 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:45.426327 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f76f945bd-csl6v_8ee834ca-5cac-4c2b-a1b2-2a9764e8489a/manager/0.log" Jul 08 05:02:45.445267 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:45.445239 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f76f945bd-csl6v_8ee834ca-5cac-4c2b-a1b2-2a9764e8489a/copy-manifests/0.log" Jul 08 05:02:46.947283 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:46.947251 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-75c8b6477f-s6gbl_fd745bbb-d45d-4f29-98db-e9d44773d9c4/manager/0.log" Jul 08 05:02:46.972243 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:46.972222 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-l9tzh_ebfe67b0-e72b-4377-a20f-7937905a92b3/openshift-lws-operator/0.log" Jul 08 05:02:47.415365 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:47.415323 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-668854d458-4ctbp_d847cbd1-645f-4266-8ad4-05ae2ac443e1/kube-auth-proxy/0.log" Jul 08 05:02:48.910025 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:48.909986 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-668854d458-4ctbp_d847cbd1-645f-4266-8ad4-05ae2ac443e1/kube-auth-proxy/0.log" Jul 08 05:02:53.580021 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:53.579992 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s4rvj_e2812fd7-bf09-4088-b1eb-67b5ac18f380/kube-multus-additional-cni-plugins/0.log" Jul 08 05:02:53.601682 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:53.601659 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s4rvj_e2812fd7-bf09-4088-b1eb-67b5ac18f380/egress-router-binary-copy/0.log" Jul 08 05:02:53.626882 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:53.626863 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s4rvj_e2812fd7-bf09-4088-b1eb-67b5ac18f380/cni-plugins/0.log" Jul 08 05:02:53.648534 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:53.648515 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s4rvj_e2812fd7-bf09-4088-b1eb-67b5ac18f380/bond-cni-plugin/0.log" Jul 08 05:02:53.669846 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:53.669830 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s4rvj_e2812fd7-bf09-4088-b1eb-67b5ac18f380/routeoverride-cni/0.log" Jul 08 05:02:53.690318 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:53.690297 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s4rvj_e2812fd7-bf09-4088-b1eb-67b5ac18f380/whereabouts-cni-bincopy/0.log" Jul 08 05:02:53.709709 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:53.709687 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s4rvj_e2812fd7-bf09-4088-b1eb-67b5ac18f380/whereabouts-cni/0.log" Jul 08 05:02:53.744182 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:53.744161 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-f545t_e6763059-f532-47e0-8523-50893eb5b3db/kube-multus/0.log" Jul 08 05:02:53.812133 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:53.812112 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6j45q_37c1fce9-b855-4864-a4e0-04b78dd27b62/network-metrics-daemon/0.log" Jul 08 05:02:53.838246 ip-10-0-133-10 kubenswrapper[2583]: I0708 05:02:53.838188 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6j45q_37c1fce9-b855-4864-a4e0-04b78dd27b62/kube-rbac-proxy/0.log"