Jul 24 13:42:46.192144 ip-10-0-128-250 systemd[1]: Starting Kubernetes Kubelet... Jul 24 13:42:46.614236 ip-10-0-128-250 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 13:42:46.614236 ip-10-0-128-250 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 24 13:42:46.614236 ip-10-0-128-250 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 13:42:46.614236 ip-10-0-128-250 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 24 13:42:46.614236 ip-10-0-128-250 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 13:42:46.615187 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.615065 2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 24 13:42:46.619339 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619317 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:42:46.619339 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619334 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:42:46.619339 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619341 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:42:46.619339 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619345 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619350 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619354 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619359 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619362 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619366 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619370 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619374 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619377 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619383 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619389 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619394 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619398 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619402 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619405 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619409 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619413 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619416 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619420 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619424 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:42:46.619601 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619428 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619433 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619437 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619441 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619444 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619448 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619451 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619455 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619459 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619462 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619466 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619470 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619474 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619479 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619484 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619489 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619493 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619497 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619501 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:42:46.620445 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619506 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619510 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619514 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619518 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619522 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619526 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619530 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619534 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619538 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619542 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619546 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619550 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619554 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619558 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619562 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619567 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619571 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619575 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619579 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619583 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:42:46.621279 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619587 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619591 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619595 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619598 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619602 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619606 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619610 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619614 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619618 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619623 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619630 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619635 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619640 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619645 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619649 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619653 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619657 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619662 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619666 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:42:46.622169 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619670 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619674 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619678 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619682 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619686 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619690 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619698 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619704 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619708 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619712 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619717 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619721 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619725 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619730 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619734 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619738 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619742 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.619746 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.620983 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.620995 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:42:46.622993 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621003 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621007 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621012 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621017 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621022 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621026 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621030 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621034 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621038 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621043 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621047 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621051 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621055 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621059 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621063 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621067 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621073 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621077 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621081 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621085 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:42:46.623847 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621090 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621094 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621100 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621107 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621111 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621116 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621120 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621124 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621128 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621132 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621137 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621141 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621145 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621150 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621153 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621160 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621167 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621172 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621177 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:42:46.624559 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621182 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621186 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621191 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621195 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621199 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621204 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621208 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621212 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621218 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621222 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621226 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621230 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621234 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621238 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621242 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621246 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621251 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621255 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621259 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621263 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:42:46.625342 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621267 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621272 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621275 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621279 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621284 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621288 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621292 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621296 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621300 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621304 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621309 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621316 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621321 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621326 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621332 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621336 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621340 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621344 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621348 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621352 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:42:46.625922 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621357 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621361 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621365 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621369 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621373 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621378 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621382 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621385 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621390 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621394 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621398 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621402 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621406 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621410 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621414 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621418 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621423 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.621427 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622123 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622133 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:42:46.626449 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622139 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622143 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622148 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622153 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622157 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622161 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622165 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622169 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622173 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622179 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622184 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622189 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622195 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622199 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622204 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622210 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622218 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622223 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622227 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622231 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:42:46.627023 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622235 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622239 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622243 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622248 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622252 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622255 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622260 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622264 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622271 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622275 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622279 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622283 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622287 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622292 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622296 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622301 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622306 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622310 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622314 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622318 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:42:46.627541 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622322 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622326 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622330 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622334 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622339 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622343 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622347 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622351 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622356 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622360 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622364 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622367 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622372 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622376 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622380 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622384 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622388 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622392 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622396 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622401 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:42:46.628090 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622407 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622412 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622420 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622425 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622430 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622435 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622439 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622443 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622448 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622452 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622456 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622461 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622465 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622469 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622473 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622479 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622483 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622487 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622491 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:42:46.628590 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622495 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622500 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622504 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622508 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622513 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622517 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622521 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622525 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622529 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622533 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622537 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622541 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622545 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622551 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622555 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622559 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622563 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.622568 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622675 2557 flags.go:64] FLAG: --address="0.0.0.0" Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622687 2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622697 2557 flags.go:64] FLAG: --anonymous-auth="true" Jul 24 13:42:46.629144 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622704 2557 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622712 2557 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622717 2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622723 2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622730 2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622735 2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622739 2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622747 2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622752 2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622757 2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622762 2557 flags.go:64] FLAG: --cgroup-root="" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622767 2557 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622772 2557 flags.go:64] FLAG: --client-ca-file="" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622777 2557 flags.go:64] FLAG: --cloud-provider="external" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622782 2557 flags.go:64] FLAG: --cluster-dns="[]" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622788 2557 flags.go:64] FLAG: --cluster-domain="" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622793 2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622797 2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622802 2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622826 2557 flags.go:64] FLAG: --container-log-max-files="5" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622832 2557 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622836 2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622841 2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 24 13:42:46.629742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622846 2557 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622853 2557 flags.go:64] FLAG: --contention-profiling="false" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622858 2557 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622862 2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622867 2557 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622872 2557 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622878 2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622883 2557 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622888 2557 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622893 2557 flags.go:64] FLAG: --enable-load-reader="false" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622898 2557 flags.go:64] FLAG: --enable-server="true" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622903 2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622910 2557 flags.go:64] FLAG: --event-burst="100" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622916 2557 flags.go:64] FLAG: --event-qps="50" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622920 2557 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622925 2557 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622932 2557 flags.go:64] FLAG: --eviction-hard="" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622939 2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622943 2557 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622948 2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622952 2557 flags.go:64] FLAG: --eviction-soft="" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622957 2557 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622962 2557 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622967 2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622972 2557 flags.go:64] FLAG: --experimental-mounter-path="" Jul 24 13:42:46.630333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622976 2557 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622981 2557 flags.go:64] FLAG: --fail-swap-on="true" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622986 2557 flags.go:64] FLAG: --feature-gates="" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622992 2557 flags.go:64] FLAG: --file-check-frequency="20s" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.622996 2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623001 2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623006 2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623010 2557 flags.go:64] FLAG: --healthz-port="10248" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623017 2557 flags.go:64] FLAG: --help="false" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623021 2557 flags.go:64] FLAG: --hostname-override="ip-10-0-128-250.ec2.internal" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623026 2557 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623031 2557 flags.go:64] FLAG: --http-check-frequency="20s" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623036 2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623042 2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623047 2557 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623052 2557 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623056 2557 flags.go:64] FLAG: --image-service-endpoint="" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623061 2557 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623065 2557 flags.go:64] FLAG: --kube-api-burst="100" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623071 2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623076 2557 flags.go:64] FLAG: --kube-api-qps="50" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623081 2557 flags.go:64] FLAG: --kube-reserved="" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623086 2557 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623091 2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 24 13:42:46.630968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623097 2557 flags.go:64] FLAG: --kubelet-cgroups="" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623101 2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623105 2557 flags.go:64] FLAG: --lock-file="" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623110 2557 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623115 2557 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623119 2557 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623128 2557 flags.go:64] FLAG: --log-json-split-stream="false" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623133 2557 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623137 2557 flags.go:64] FLAG: --log-text-split-stream="false" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623142 2557 flags.go:64] FLAG: --logging-format="text" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623147 2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623153 2557 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623157 2557 flags.go:64] FLAG: --manifest-url="" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623162 2557 flags.go:64] FLAG: --manifest-url-header="" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623169 2557 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623174 2557 flags.go:64] FLAG: --max-open-files="1000000" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623182 2557 flags.go:64] FLAG: --max-pods="110" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623187 2557 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623191 2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623196 2557 flags.go:64] FLAG: --memory-manager-policy="None" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623201 2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623206 2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623211 2557 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623215 2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623226 2557 flags.go:64] FLAG: --node-status-max-images="50" Jul 24 13:42:46.631586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623230 2557 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623235 2557 flags.go:64] FLAG: --oom-score-adj="-999" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623240 2557 flags.go:64] FLAG: --pod-cidr="" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623246 2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623254 2557 flags.go:64] FLAG: --pod-manifest-path="" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623259 2557 flags.go:64] FLAG: --pod-max-pids="-1" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623266 2557 flags.go:64] FLAG: --pods-per-core="0" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623271 2557 flags.go:64] FLAG: --port="10250" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623275 2557 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623280 2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0dd709f7b61334bfd" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623285 2557 flags.go:64] FLAG: --qos-reserved="" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623290 2557 flags.go:64] FLAG: --read-only-port="10255" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623295 2557 flags.go:64] FLAG: --register-node="true" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623299 2557 flags.go:64] FLAG: --register-with-taints="" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623305 2557 flags.go:64] FLAG: --registry-burst="10" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623310 2557 flags.go:64] FLAG: --registry-qps="5" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623314 2557 flags.go:64] FLAG: --reserved-cpus="" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623319 2557 flags.go:64] FLAG: --reserved-memory="" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623325 2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623329 2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623334 2557 flags.go:64] FLAG: --rotate-certificates="false" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623339 2557 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623344 2557 flags.go:64] FLAG: --runonce="false" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623348 2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623355 2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 24 13:42:46.632222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623360 2557 flags.go:64] FLAG: --seccomp-default="false" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623364 2557 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623369 2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623374 2557 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623379 2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623383 2557 flags.go:64] FLAG: --storage-driver-password="root" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623388 2557 flags.go:64] FLAG: --storage-driver-secure="false" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623393 2557 flags.go:64] FLAG: --storage-driver-table="stats" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623397 2557 flags.go:64] FLAG: --storage-driver-user="root" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623402 2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623407 2557 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623412 2557 flags.go:64] FLAG: --system-cgroups="" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623417 2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623427 2557 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623431 2557 flags.go:64] FLAG: --tls-cert-file="" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623436 2557 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623442 2557 flags.go:64] FLAG: --tls-min-version="" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623447 2557 flags.go:64] FLAG: --tls-private-key-file="" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623452 2557 flags.go:64] FLAG: --topology-manager-policy="none" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623456 2557 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623461 2557 flags.go:64] FLAG: --topology-manager-scope="container" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623465 2557 flags.go:64] FLAG: --v="2" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623472 2557 flags.go:64] FLAG: --version="false" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623478 2557 flags.go:64] FLAG: --vmodule="" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623484 2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 24 13:42:46.632874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.623489 2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623674 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623681 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623685 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623690 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623697 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623704 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623709 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623714 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623718 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623723 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623727 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623732 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623736 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623740 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623744 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623748 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623752 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623757 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:42:46.633490 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623762 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623768 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623772 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623776 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623781 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623785 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623789 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623793 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623797 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623801 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623823 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623828 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623832 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623836 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623840 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623844 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623848 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623852 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623855 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623861 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:42:46.634060 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623865 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623869 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623873 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623876 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623881 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623884 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623888 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623892 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623896 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623901 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623905 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623909 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623914 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623920 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623924 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623928 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623933 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623937 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623942 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:42:46.634547 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623948 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623954 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623959 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623964 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623968 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623972 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623977 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623982 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623986 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623990 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623994 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.623999 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624005 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624019 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624024 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624030 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624037 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624041 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624045 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624050 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:42:46.635037 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624054 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624058 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624063 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624067 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624071 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624075 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624080 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624086 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624090 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624094 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624098 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624102 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624106 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624110 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624115 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624119 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624123 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624127 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624131 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624135 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:42:46.635526 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624139 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:42:46.636040 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.624143 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:42:46.636040 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.624152 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 13:42:46.636040 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.631247 2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 24 13:42:46.636040 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.631262 2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 24 13:42:46.636040 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.631278 2557 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 24 13:42:46.636040 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.631284 2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 24 13:42:46.636040 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631345 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:42:46.636040 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631350 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:42:46.636040 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631353 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:42:46.636040 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631356 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:42:46.636040 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631360 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:42:46.636040 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631363 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:42:46.636040 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631366 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:42:46.636040 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631368 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631371 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631374 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631377 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631379 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631390 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631394 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631396 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631399 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631402 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631404 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631407 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631409 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631417 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631420 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631422 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631425 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631427 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631430 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631432 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:42:46.636409 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631435 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631438 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631441 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631444 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631446 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631449 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631452 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631454 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631457 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631459 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631462 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631464 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631466 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631469 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631471 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631473 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631476 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631478 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631487 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631490 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:42:46.636995 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631492 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631495 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631497 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631499 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631502 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631505 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631507 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631509 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631512 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631514 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631518 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631522 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631526 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631529 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631533 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631535 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631538 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631541 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631544 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:42:46.637527 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631546 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631549 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631551 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631554 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631558 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631561 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631564 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631566 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631569 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631572 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631574 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631577 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631584 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631587 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631589 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631592 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631594 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631597 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631600 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631602 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:42:46.638012 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631604 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:42:46.638512 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631607 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:42:46.638512 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631609 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:42:46.638512 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631611 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:42:46.638512 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631614 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:42:46.638512 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631616 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:42:46.638512 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631619 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:42:46.638512 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631622 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:42:46.638512 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631624 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:42:46.638512 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631627 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:42:46.638512 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631630 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:42:46.638512 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631632 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:42:46.638512 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631634 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:42:46.638512 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.631639 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 13:42:46.638512 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631795 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:42:46.638512 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631800 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631802 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631824 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631827 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631830 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631832 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631834 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631837 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631839 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631842 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631850 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631853 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631856 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631858 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631860 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631863 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631865 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631868 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631870 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631873 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:42:46.638962 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631875 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631878 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631881 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631884 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631886 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631889 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631891 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631893 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631897 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631900 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631902 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631905 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631907 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631910 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631912 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631915 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631917 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631919 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631921 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631924 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:42:46.639685 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631926 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631929 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631931 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631940 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631943 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631945 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631948 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631950 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631952 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631955 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631957 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631959 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631962 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631964 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631967 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631969 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631972 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631974 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631977 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631979 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:42:46.640229 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631982 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631984 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631987 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631989 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631991 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631994 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631996 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.631999 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632002 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632005 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632008 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632011 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632013 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632015 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632018 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632020 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632029 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632032 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632034 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632036 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:42:46.640729 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632039 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632041 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632043 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632046 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632050 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632053 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632056 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632058 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632061 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632064 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632066 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632069 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632071 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632073 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632076 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632078 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632081 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:42:46.641268 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:46.632083 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:42:46.641687 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.632089 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 13:42:46.641687 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.632200 2557 server.go:962] "Client rotation is on, will bootstrap in background" Jul 24 13:42:46.641687 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.634168 2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 24 13:42:46.641687 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.635255 2557 server.go:1019] "Starting client certificate rotation" Jul 24 13:42:46.641687 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.635360 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 13:42:46.641687 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.636349 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 13:42:46.641687 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.640006 2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 13:42:46.641687 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.641193 2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 13:42:46.652490 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.652454 2557 log.go:25] "Validated CRI v1 runtime API" Jul 24 13:42:46.657673 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.657655 2557 log.go:25] "Validated CRI v1 image API" Jul 24 13:42:46.658527 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.658511 2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 24 13:42:46.660745 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.660718 2557 fs.go:135] Filesystem UUIDs: map[401197f9-1922-43c0-a368-8078f638e33d:/dev/nvme0n1p3 4eca0b37-b24a-4911-aad8-762795b4e73f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 24 13:42:46.660835 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.660745 2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 24 13:42:46.665193 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.665085 2557 manager.go:216] Machine: {Timestamp:2026-07-24 13:42:46.664580222 +0000 UTC m=+0.351341629 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099525 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b00f0837ff9ed496580b0aeeeb748 SystemUUID:ec2b00f0-837f-f9ed-4965-80b0aeeeb748 BootID:d0e1df40-f48b-4c57-83bd-07053ddeb605 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:30:70:ea:e8:89 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:30:70:ea:e8:89 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:6f:82:d5:08:43 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 24 13:42:46.665193 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.665191 2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 24 13:42:46.665295 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.665269 2557 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 24 13:42:46.668215 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.668184 2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 24 13:42:46.668393 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.668218 2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-250.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 24 13:42:46.668775 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.668761 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 13:42:46.669468 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.669454 2557 topology_manager.go:138] "Creating topology manager with none policy" Jul 24 13:42:46.669532 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.669475 2557 container_manager_linux.go:309] "Creating device plugin manager" Jul 24 13:42:46.669532 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.669486 2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 13:42:46.670349 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.670336 2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 13:42:46.671733 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.671720 2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 24 13:42:46.674557 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.674543 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 24 13:42:46.674746 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.674734 2557 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 24 13:42:46.674830 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.674773 2557 kubelet.go:486] "Attempting to sync node with API server" Jul 24 13:42:46.674830 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.674789 2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 24 13:42:46.674830 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.674825 2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 24 13:42:46.674964 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.674906 2557 kubelet.go:398] "Adding apiserver pod source" Jul 24 13:42:46.674964 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.674920 2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 24 13:42:46.675360 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.675346 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 13:42:46.675425 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.675371 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 13:42:46.677226 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.677208 2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 24 13:42:46.680752 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.680733 2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 24 13:42:46.680853 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.680773 2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 24 13:42:46.682269 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.682255 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 24 13:42:46.682371 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.682275 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 24 13:42:46.682371 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.682284 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 24 13:42:46.682371 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.682293 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 24 13:42:46.682371 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.682304 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 24 13:42:46.682371 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.682312 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 24 13:42:46.682371 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.682323 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 24 13:42:46.682371 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.682330 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 24 13:42:46.682371 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.682340 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 24 13:42:46.682371 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.682348 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 24 13:42:46.682371 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.682360 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 24 13:42:46.682371 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.682368 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 24 13:42:46.683387 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.683376 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 24 13:42:46.683446 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.683390 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 24 13:42:46.685860 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.685837 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-250.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 24 13:42:46.685952 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.685928 2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-250.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 24 13:42:46.686039 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.686021 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 24 13:42:46.686879 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.686866 2557 server.go:1268] "Started kubelet" Jul 24 13:42:46.686967 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.686945 2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 24 13:42:46.687031 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.686952 2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 24 13:42:46.687068 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.687030 2557 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 24 13:42:46.687768 ip-10-0-128-250 systemd[1]: Started Kubernetes Kubelet. Jul 24 13:42:46.688329 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.688232 2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 24 13:42:46.689100 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.689088 2557 server.go:310] "Adding debug handlers to kubelet server" Jul 24 13:42:46.692472 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.692452 2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 24 13:42:46.693020 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.692205 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-250.ec2.internal.18c53d9bccc330b0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-250.ec2.internal,UID:ip-10-0-128-250.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-250.ec2.internal,},FirstTimestamp:2026-07-24 13:42:46.68683896 +0000 UTC m=+0.373600369,LastTimestamp:2026-07-24 13:42:46.68683896 +0000 UTC m=+0.373600369,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-250.ec2.internal,}" Jul 24 13:42:46.694548 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.694533 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bbwj6" Jul 24 13:42:46.695306 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.695292 2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 24 13:42:46.695365 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.695296 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 24 13:42:46.695435 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.695409 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:46.695953 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.695937 2557 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 24 13:42:46.696061 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.696050 2557 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 24 13:42:46.696118 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.695912 2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 24 13:42:46.696197 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.696183 2557 reconstruct.go:97] "Volume reconstruction finished" Jul 24 13:42:46.696263 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.696250 2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 24 13:42:46.696324 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.696263 2557 factory.go:55] Registering systemd factory Jul 24 13:42:46.696324 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.696275 2557 factory.go:223] Registration of the systemd container factory successfully Jul 24 13:42:46.696505 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.696494 2557 factory.go:153] Registering CRI-O factory Jul 24 13:42:46.696580 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.696507 2557 factory.go:223] Registration of the crio container factory successfully Jul 24 13:42:46.696580 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.696518 2557 reconciler.go:29] "Reconciler: start to sync state" Jul 24 13:42:46.696580 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.696529 2557 factory.go:103] Registering Raw factory Jul 24 13:42:46.696580 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.696542 2557 manager.go:1196] Started watching for new ooms in manager Jul 24 13:42:46.697818 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.697786 2557 manager.go:349] Starting recovery of all containers Jul 24 13:42:46.701145 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.701124 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bbwj6" Jul 24 13:42:46.710565 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.710530 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 24 13:42:46.710676 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.710583 2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-250.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 24 13:42:46.717153 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.717129 2557 manager.go:354] Recovery completed Jul 24 13:42:46.722500 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.722482 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:42:46.724984 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.724966 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:42:46.725085 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.724996 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:42:46.725085 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.725006 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:42:46.725552 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.725535 2557 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 24 13:42:46.725552 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.725548 2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 24 13:42:46.725552 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.725550 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:42:46.725552 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.725564 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 24 13:42:46.726264 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.726250 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:42:46.726319 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.726283 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:42:46.726319 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.726299 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:42:46.726746 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.726673 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-250.ec2.internal.18c53d9bcf093786 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-250.ec2.internal,UID:ip-10-0-128-250.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-250.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-250.ec2.internal,},FirstTimestamp:2026-07-24 13:42:46.724982662 +0000 UTC m=+0.411744071,LastTimestamp:2026-07-24 13:42:46.724982662 +0000 UTC m=+0.411744071,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-250.ec2.internal,}" Jul 24 13:42:46.727032 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.727015 2557 policy_none.go:49] "None policy: Start" Jul 24 13:42:46.727114 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.727036 2557 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 24 13:42:46.727114 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.727051 2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 24 13:42:46.728174 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.728164 2557 policy_none.go:47] "Start" Jul 24 13:42:46.763443 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.763425 2557 manager.go:337] "Starting Device Plugin manager" Jul 24 13:42:46.763553 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.763473 2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 24 13:42:46.763553 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.763489 2557 server.go:87] "Starting device plugin registration server" Jul 24 13:42:46.763749 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.763738 2557 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 24 13:42:46.763780 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.763751 2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 24 13:42:46.763928 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.763916 2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 24 13:42:46.763998 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.763988 2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 24 13:42:46.763998 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.763996 2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 24 13:42:46.764492 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.764470 2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 24 13:42:46.764575 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.764517 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:46.784290 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.784264 2557 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2591 into the system container \"/system.slice\": " Jul 24 13:42:46.864349 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.864278 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:42:46.866263 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.866241 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:42:46.866347 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.866282 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:42:46.866347 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.866293 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:42:46.866347 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.866319 2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-250.ec2.internal" Jul 24 13:42:46.869838 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.869787 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 24 13:42:46.871149 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.871131 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 24 13:42:46.871239 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.871174 2557 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 24 13:42:46.871239 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.871198 2557 kubelet.go:2448] "Starting kubelet main sync loop" Jul 24 13:42:46.871239 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.871235 2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 24 13:42:46.872756 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.872734 2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-250.ec2.internal" Jul 24 13:42:46.872858 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.872763 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-250.ec2.internal\": node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:46.873950 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.873932 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:42:46.899171 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.899143 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:46.971472 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.971429 2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-250.ec2.internal"] Jul 24 13:42:46.971572 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.971532 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:42:46.972537 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.972518 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:42:46.972611 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.972553 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:42:46.972611 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.972567 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:42:46.974074 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.974058 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:42:46.974196 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.974180 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal" Jul 24 13:42:46.974243 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.974216 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:42:46.974824 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.974795 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:42:46.974893 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.974837 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:42:46.974893 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.974852 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:42:46.974893 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.974802 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:42:46.974983 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.974914 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:42:46.974983 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.974927 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:42:46.976092 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.976077 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-250.ec2.internal" Jul 24 13:42:46.976135 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.976103 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:42:46.977986 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.977970 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:42:46.978058 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.978000 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:42:46.978058 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.978013 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:42:46.995423 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.995401 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-250.ec2.internal\" not found" node="ip-10-0-128-250.ec2.internal" Jul 24 13:42:46.997441 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.997422 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c126e9785f9c1e44eb018143a4d59ea2-config\") pod \"kube-apiserver-proxy-ip-10-0-128-250.ec2.internal\" (UID: \"c126e9785f9c1e44eb018143a4d59ea2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-250.ec2.internal" Jul 24 13:42:46.997556 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.997445 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f08d9cec52ff7c0389f134ed4757b337-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal\" (UID: \"f08d9cec52ff7c0389f134ed4757b337\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal" Jul 24 13:42:46.997556 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:46.997470 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f08d9cec52ff7c0389f134ed4757b337-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal\" (UID: \"f08d9cec52ff7c0389f134ed4757b337\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal" Jul 24 13:42:46.999232 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.999216 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:46.999381 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:46.999364 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-250.ec2.internal\" not found" node="ip-10-0-128-250.ec2.internal" Jul 24 13:42:47.098109 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.098081 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f08d9cec52ff7c0389f134ed4757b337-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal\" (UID: \"f08d9cec52ff7c0389f134ed4757b337\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal" Jul 24 13:42:47.098109 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.098111 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f08d9cec52ff7c0389f134ed4757b337-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal\" (UID: \"f08d9cec52ff7c0389f134ed4757b337\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal" Jul 24 13:42:47.098109 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.098129 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c126e9785f9c1e44eb018143a4d59ea2-config\") pod \"kube-apiserver-proxy-ip-10-0-128-250.ec2.internal\" (UID: \"c126e9785f9c1e44eb018143a4d59ea2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-250.ec2.internal" Jul 24 13:42:47.098372 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.098175 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c126e9785f9c1e44eb018143a4d59ea2-config\") pod \"kube-apiserver-proxy-ip-10-0-128-250.ec2.internal\" (UID: \"c126e9785f9c1e44eb018143a4d59ea2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-250.ec2.internal" Jul 24 13:42:47.098372 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.098183 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f08d9cec52ff7c0389f134ed4757b337-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal\" (UID: \"f08d9cec52ff7c0389f134ed4757b337\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal" Jul 24 13:42:47.098372 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.098198 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f08d9cec52ff7c0389f134ed4757b337-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal\" (UID: \"f08d9cec52ff7c0389f134ed4757b337\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal" Jul 24 13:42:47.100194 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:47.100179 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:47.200491 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:47.200410 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:47.297792 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.297767 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal" Jul 24 13:42:47.301502 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:47.301483 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:47.303654 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.303640 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-250.ec2.internal" Jul 24 13:42:47.401726 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:47.401690 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:47.502346 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:47.502267 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:47.570610 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.570580 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:42:47.602552 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:47.602530 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:47.605168 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.605152 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:42:47.635380 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.635356 2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 24 13:42:47.635881 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.635495 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 13:42:47.635881 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.635522 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 13:42:47.635881 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.635532 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 13:42:47.703500 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:47.703474 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:47.712614 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.712576 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 13:37:46 +0000 UTC" deadline="2028-02-22 17:13:10.013158004 +0000 UTC" Jul 24 13:42:47.712614 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.712614 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13875h30m22.300549261s" Jul 24 13:42:47.804247 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:47.804222 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:47.806486 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:47.806449 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf08d9cec52ff7c0389f134ed4757b337.slice/crio-40a03cc162028cb8313e398fd12e565406a653209bb341583c8b642607a057fa WatchSource:0}: Error finding container 40a03cc162028cb8313e398fd12e565406a653209bb341583c8b642607a057fa: Status 404 returned error can't find the container with id 40a03cc162028cb8313e398fd12e565406a653209bb341583c8b642607a057fa Jul 24 13:42:47.807225 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:47.807203 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc126e9785f9c1e44eb018143a4d59ea2.slice/crio-8c6a1c3193b8e06ac4ae0e87252f0e2aa8ef79b7e308cbf808dc1496d1149aad WatchSource:0}: Error finding container 8c6a1c3193b8e06ac4ae0e87252f0e2aa8ef79b7e308cbf808dc1496d1149aad: Status 404 returned error can't find the container with id 8c6a1c3193b8e06ac4ae0e87252f0e2aa8ef79b7e308cbf808dc1496d1149aad Jul 24 13:42:47.811732 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.811710 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:42:47.877522 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.877464 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal" event={"ID":"f08d9cec52ff7c0389f134ed4757b337","Type":"ContainerStarted","Data":"40a03cc162028cb8313e398fd12e565406a653209bb341583c8b642607a057fa"} Jul 24 13:42:47.878625 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:47.878601 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-250.ec2.internal" event={"ID":"c126e9785f9c1e44eb018143a4d59ea2","Type":"ContainerStarted","Data":"8c6a1c3193b8e06ac4ae0e87252f0e2aa8ef79b7e308cbf808dc1496d1149aad"} Jul 24 13:42:47.905149 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:47.905118 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:48.005618 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:48.005588 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:48.106218 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:48.106135 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:48.206984 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:48.206954 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-250.ec2.internal\" not found" Jul 24 13:42:48.229990 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.229969 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:42:48.295723 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.295693 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal" Jul 24 13:42:48.307038 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.306915 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 13:42:48.308242 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.308001 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-250.ec2.internal" Jul 24 13:42:48.316242 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.316154 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 13:42:48.676413 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.676388 2557 apiserver.go:52] "Watching apiserver" Jul 24 13:42:48.682622 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.682599 2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 24 13:42:48.683060 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.683037 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-p9gvn","openshift-dns/node-resolver-5mnw5","openshift-multus/network-metrics-daemon-vjcfb","openshift-network-diagnostics/network-check-target-9sfhz","openshift-network-operator/iptables-alerter-9v9sk","kube-system/konnectivity-agent-tqkzp","openshift-image-registry/node-ca-j5cfd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal","openshift-multus/multus-additional-cni-plugins-lmwnz","openshift-multus/multus-fdrqb","openshift-ovn-kubernetes/ovnkube-node-h8gk4","kube-system/global-pull-secret-syncer-q7hvp","kube-system/kube-apiserver-proxy-ip-10-0-128-250.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf"] Jul 24 13:42:48.684511 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.684491 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j5cfd" Jul 24 13:42:48.686143 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.686124 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.686509 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.686456 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 24 13:42:48.686592 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.686558 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-24ckj\"" Jul 24 13:42:48.686865 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.686842 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 24 13:42:48.686937 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.686841 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 24 13:42:48.687160 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.687136 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:42:48.687230 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:48.687199 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:42:48.688055 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.688036 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-l5lpk\"" Jul 24 13:42:48.688246 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.688231 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 24 13:42:48.688317 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.688272 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 24 13:42:48.688624 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.688601 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 24 13:42:48.688712 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.688680 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 24 13:42:48.688822 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.688785 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 24 13:42:48.688993 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.688977 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:42:48.689051 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.689011 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 24 13:42:48.689051 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:48.689039 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:42:48.690094 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.690073 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9v9sk" Jul 24 13:42:48.691408 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.691185 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tqkzp" Jul 24 13:42:48.691791 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.691772 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:42:48.691922 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.691828 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 24 13:42:48.691922 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.691916 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 24 13:42:48.692024 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.691998 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tnx5g\"" Jul 24 13:42:48.692511 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.692494 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.693029 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.693012 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zzl6s\"" Jul 24 13:42:48.693115 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.693042 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 24 13:42:48.693115 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.693096 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 24 13:42:48.694124 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.694106 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.694565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.694547 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 24 13:42:48.694916 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.694899 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 24 13:42:48.695057 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.695036 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 24 13:42:48.695139 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.695100 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 24 13:42:48.695139 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.695104 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hxkxm\"" Jul 24 13:42:48.695249 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.695231 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 24 13:42:48.695644 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.695630 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 24 13:42:48.695862 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.695847 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 24 13:42:48.696135 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.696114 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 24 13:42:48.696209 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.696137 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h8vnj\"" Jul 24 13:42:48.696540 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.696392 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 24 13:42:48.697569 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.697552 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5mnw5" Jul 24 13:42:48.703230 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.703210 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 24 13:42:48.703593 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.703573 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qm7f8\"" Jul 24 13:42:48.704703 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.704111 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.706960 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-ovn-node-metrics-cert\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.706998 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707031 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3f4a0f2f-93e0-40f6-9ec7-6b61a09ade21-agent-certs\") pod \"konnectivity-agent-tqkzp\" (UID: \"3f4a0f2f-93e0-40f6-9ec7-6b61a09ade21\") " pod="kube-system/konnectivity-agent-tqkzp" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707061 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3f4a0f2f-93e0-40f6-9ec7-6b61a09ade21-konnectivity-ca\") pod \"konnectivity-agent-tqkzp\" (UID: \"3f4a0f2f-93e0-40f6-9ec7-6b61a09ade21\") " pod="kube-system/konnectivity-agent-tqkzp" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707099 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6df914a7-aff6-4e3a-936b-a8852546642f-cnibin\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707102 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707163 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6df914a7-aff6-4e3a-936b-a8852546642f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707204 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-run-systemd\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707231 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdk5n\" (UniqueName: \"kubernetes.io/projected/46030f6f-52fc-40f7-b5d4-7febe8bd998a-kube-api-access-xdk5n\") pod \"network-metrics-daemon-vjcfb\" (UID: \"46030f6f-52fc-40f7-b5d4-7febe8bd998a\") " pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707260 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp2nb\" (UniqueName: \"kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb\") pod \"network-check-target-9sfhz\" (UID: \"f311e199-3ddf-4650-8f2b-cde419d63d1a\") " pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707290 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9df22139-a68a-421b-807c-e6afb4bb691e-serviceca\") pod \"node-ca-j5cfd\" (UID: \"9df22139-a68a-421b-807c-e6afb4bb691e\") " pod="openshift-image-registry/node-ca-j5cfd" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707318 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6df914a7-aff6-4e3a-936b-a8852546642f-os-release\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707349 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6df914a7-aff6-4e3a-936b-a8852546642f-cni-binary-copy\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707379 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9jwj\" (UniqueName: \"kubernetes.io/projected/5513d77f-a9df-4125-bd63-d8f9df7b7e93-kube-api-access-w9jwj\") pod \"node-resolver-5mnw5\" (UID: \"5513d77f-a9df-4125-bd63-d8f9df7b7e93\") " pod="openshift-dns/node-resolver-5mnw5" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707462 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-systemd-units\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707489 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-env-overrides\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.707859 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707550 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-sys-fs\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.708565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707607 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvlq4\" (UniqueName: \"kubernetes.io/projected/967e0e9f-301b-4f3c-ab27-df6f3cff68ee-kube-api-access-qvlq4\") pod \"iptables-alerter-9v9sk\" (UID: \"967e0e9f-301b-4f3c-ab27-df6f3cff68ee\") " pod="openshift-network-operator/iptables-alerter-9v9sk" Jul 24 13:42:48.708565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707642 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6df914a7-aff6-4e3a-936b-a8852546642f-system-cni-dir\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.708565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707673 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-etc-openvswitch\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.708565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707707 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-ovnkube-config\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.708565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707746 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-etc-selinux\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.708565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.707786 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-device-dir\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.708565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708153 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56j8q\" (UniqueName: \"kubernetes.io/projected/9df22139-a68a-421b-807c-e6afb4bb691e-kube-api-access-56j8q\") pod \"node-ca-j5cfd\" (UID: \"9df22139-a68a-421b-807c-e6afb4bb691e\") " pod="openshift-image-registry/node-ca-j5cfd" Jul 24 13:42:48.708565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708181 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/967e0e9f-301b-4f3c-ab27-df6f3cff68ee-host-slash\") pod \"iptables-alerter-9v9sk\" (UID: \"967e0e9f-301b-4f3c-ab27-df6f3cff68ee\") " pod="openshift-network-operator/iptables-alerter-9v9sk" Jul 24 13:42:48.708565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708194 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-cni-netd\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.708565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708207 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wlmg\" (UniqueName: \"kubernetes.io/projected/6df914a7-aff6-4e3a-936b-a8852546642f-kube-api-access-8wlmg\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.708565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708224 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5513d77f-a9df-4125-bd63-d8f9df7b7e93-tmp-dir\") pod \"node-resolver-5mnw5\" (UID: \"5513d77f-a9df-4125-bd63-d8f9df7b7e93\") " pod="openshift-dns/node-resolver-5mnw5" Jul 24 13:42:48.708565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708236 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-run-ovn\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.708565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708248 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-node-log\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.708565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708261 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-cni-bin\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.708565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708297 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs\") pod \"network-metrics-daemon-vjcfb\" (UID: \"46030f6f-52fc-40f7-b5d4-7febe8bd998a\") " pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:42:48.708565 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708311 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcsxg\" (UniqueName: \"kubernetes.io/projected/043101ce-c15f-4359-95d1-0432fd3498b6-kube-api-access-tcsxg\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.709427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708326 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-kubelet\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.709427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708339 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-run-openvswitch\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.709427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708352 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-run-ovn-kubernetes\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.709427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708364 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-ovnkube-script-lib\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.709427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708377 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-socket-dir\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.709427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708389 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9df22139-a68a-421b-807c-e6afb4bb691e-host\") pod \"node-ca-j5cfd\" (UID: \"9df22139-a68a-421b-807c-e6afb4bb691e\") " pod="openshift-image-registry/node-ca-j5cfd" Jul 24 13:42:48.709427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708401 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/967e0e9f-301b-4f3c-ab27-df6f3cff68ee-iptables-alerter-script\") pod \"iptables-alerter-9v9sk\" (UID: \"967e0e9f-301b-4f3c-ab27-df6f3cff68ee\") " pod="openshift-network-operator/iptables-alerter-9v9sk" Jul 24 13:42:48.709427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708419 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6df914a7-aff6-4e3a-936b-a8852546642f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.709427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708435 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6df914a7-aff6-4e3a-936b-a8852546642f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.709427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708452 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5513d77f-a9df-4125-bd63-d8f9df7b7e93-hosts-file\") pod \"node-resolver-5mnw5\" (UID: \"5513d77f-a9df-4125-bd63-d8f9df7b7e93\") " pod="openshift-dns/node-resolver-5mnw5" Jul 24 13:42:48.709427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708465 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-log-socket\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.709427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708481 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.709427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708498 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhssd\" (UniqueName: \"kubernetes.io/projected/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-kube-api-access-nhssd\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.709427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708510 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-registration-dir\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.709427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708525 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-slash\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.709427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708538 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-run-netns\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.710135 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.708556 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-var-lib-openvswitch\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.711036 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.711011 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 24 13:42:48.711123 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.711088 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 13:42:48.711275 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.711260 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5b9zc\"" Jul 24 13:42:48.712065 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.712051 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.713251 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.713235 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:48.713322 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:48.713286 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:42:48.714140 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.714125 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:42:48.714140 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.714135 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 24 13:42:48.714273 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.714140 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-45j66\"" Jul 24 13:42:48.725013 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.724996 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-msrz6" Jul 24 13:42:48.732154 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.732130 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-msrz6" Jul 24 13:42:48.796986 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.796960 2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 24 13:42:48.808767 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.808737 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-systemd\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.808935 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.808777 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret\") pod \"global-pull-secret-syncer-q7hvp\" (UID: \"533b42cc-ae61-4729-b5a1-e904f90f7fcc\") " pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:48.808935 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.808829 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6df914a7-aff6-4e3a-936b-a8852546642f-cnibin\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.808935 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.808853 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-run-systemd\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.808935 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.808889 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdk5n\" (UniqueName: \"kubernetes.io/projected/46030f6f-52fc-40f7-b5d4-7febe8bd998a-kube-api-access-xdk5n\") pod \"network-metrics-daemon-vjcfb\" (UID: \"46030f6f-52fc-40f7-b5d4-7febe8bd998a\") " pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:42:48.808935 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.808912 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9df22139-a68a-421b-807c-e6afb4bb691e-serviceca\") pod \"node-ca-j5cfd\" (UID: \"9df22139-a68a-421b-807c-e6afb4bb691e\") " pod="openshift-image-registry/node-ca-j5cfd" Jul 24 13:42:48.809192 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.808937 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-sysctl-conf\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.809192 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.808947 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-run-systemd\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.809192 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.808958 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-var-lib-kubelet\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.809192 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.808948 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6df914a7-aff6-4e3a-936b-a8852546642f-cnibin\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.809192 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.808996 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6df914a7-aff6-4e3a-936b-a8852546642f-os-release\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.809192 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809025 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6df914a7-aff6-4e3a-936b-a8852546642f-cni-binary-copy\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.809477 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809295 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6df914a7-aff6-4e3a-936b-a8852546642f-os-release\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.809477 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809326 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9df22139-a68a-421b-807c-e6afb4bb691e-serviceca\") pod \"node-ca-j5cfd\" (UID: \"9df22139-a68a-421b-807c-e6afb4bb691e\") " pod="openshift-image-registry/node-ca-j5cfd" Jul 24 13:42:48.809477 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809334 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9jwj\" (UniqueName: \"kubernetes.io/projected/5513d77f-a9df-4125-bd63-d8f9df7b7e93-kube-api-access-w9jwj\") pod \"node-resolver-5mnw5\" (UID: \"5513d77f-a9df-4125-bd63-d8f9df7b7e93\") " pod="openshift-dns/node-resolver-5mnw5" Jul 24 13:42:48.809477 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809370 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-sys-fs\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.809477 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809414 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6df914a7-aff6-4e3a-936b-a8852546642f-cni-binary-copy\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.809477 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809458 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-system-cni-dir\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.809477 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809475 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-run-multus-certs\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.809783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809508 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10d661f4-051c-4c4d-b80b-bb75aff5161d-tmp\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.809783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809524 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-ovnkube-config\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.809783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809557 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-run-netns\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.809783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809568 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-sys-fs\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.809783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809675 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-run\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.809783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809700 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvbtr\" (UniqueName: \"kubernetes.io/projected/10d661f4-051c-4c4d-b80b-bb75aff5161d-kube-api-access-vvbtr\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.809783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809737 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/967e0e9f-301b-4f3c-ab27-df6f3cff68ee-host-slash\") pod \"iptables-alerter-9v9sk\" (UID: \"967e0e9f-301b-4f3c-ab27-df6f3cff68ee\") " pod="openshift-network-operator/iptables-alerter-9v9sk" Jul 24 13:42:48.809783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809762 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3f4a0f2f-93e0-40f6-9ec7-6b61a09ade21-agent-certs\") pod \"konnectivity-agent-tqkzp\" (UID: \"3f4a0f2f-93e0-40f6-9ec7-6b61a09ade21\") " pod="kube-system/konnectivity-agent-tqkzp" Jul 24 13:42:48.809783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809782 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5513d77f-a9df-4125-bd63-d8f9df7b7e93-tmp-dir\") pod \"node-resolver-5mnw5\" (UID: \"5513d77f-a9df-4125-bd63-d8f9df7b7e93\") " pod="openshift-dns/node-resolver-5mnw5" Jul 24 13:42:48.810221 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809920 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/967e0e9f-301b-4f3c-ab27-df6f3cff68ee-host-slash\") pod \"iptables-alerter-9v9sk\" (UID: \"967e0e9f-301b-4f3c-ab27-df6f3cff68ee\") " pod="openshift-network-operator/iptables-alerter-9v9sk" Jul 24 13:42:48.810221 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.809976 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-node-log\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.810221 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810003 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-cni-bin\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.810221 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810020 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-ovnkube-config\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.810221 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810028 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs\") pod \"network-metrics-daemon-vjcfb\" (UID: \"46030f6f-52fc-40f7-b5d4-7febe8bd998a\") " pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:42:48.810221 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810043 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5513d77f-a9df-4125-bd63-d8f9df7b7e93-tmp-dir\") pod \"node-resolver-5mnw5\" (UID: \"5513d77f-a9df-4125-bd63-d8f9df7b7e93\") " pod="openshift-dns/node-resolver-5mnw5" Jul 24 13:42:48.810221 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810058 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcsxg\" (UniqueName: \"kubernetes.io/projected/043101ce-c15f-4359-95d1-0432fd3498b6-kube-api-access-tcsxg\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.810221 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810070 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-node-log\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.810221 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810082 2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 24 13:42:48.810221 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810081 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-cni-bin\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.810221 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810089 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c799cb15-d6c2-4140-9af0-64b652841355-cni-binary-copy\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.810221 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810135 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-var-lib-cni-bin\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.810221 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:48.810147 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:42:48.810221 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810158 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-os-release\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.810221 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:48.810195 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs podName:46030f6f-52fc-40f7-b5d4-7febe8bd998a nodeName:}" failed. No retries permitted until 2026-07-24 13:42:49.310179763 +0000 UTC m=+2.996941172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs") pod "network-metrics-daemon-vjcfb" (UID: "46030f6f-52fc-40f7-b5d4-7febe8bd998a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:42:48.810221 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810212 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-kubelet\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810258 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-run-ovn-kubernetes\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810295 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-ovnkube-script-lib\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810317 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/967e0e9f-301b-4f3c-ab27-df6f3cff68ee-iptables-alerter-script\") pod \"iptables-alerter-9v9sk\" (UID: \"967e0e9f-301b-4f3c-ab27-df6f3cff68ee\") " pod="openshift-network-operator/iptables-alerter-9v9sk" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810325 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-run-ovn-kubernetes\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810295 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-kubelet\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810338 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-run-k8s-cni-cncf-io\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810385 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-sysconfig\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810420 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5513d77f-a9df-4125-bd63-d8f9df7b7e93-hosts-file\") pod \"node-resolver-5mnw5\" (UID: \"5513d77f-a9df-4125-bd63-d8f9df7b7e93\") " pod="openshift-dns/node-resolver-5mnw5" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810495 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810525 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5513d77f-a9df-4125-bd63-d8f9df7b7e93-hosts-file\") pod \"node-resolver-5mnw5\" (UID: \"5513d77f-a9df-4125-bd63-d8f9df7b7e93\") " pod="openshift-dns/node-resolver-5mnw5" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810539 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhssd\" (UniqueName: \"kubernetes.io/projected/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-kube-api-access-nhssd\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810556 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-lib-modules\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810568 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810578 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/533b42cc-ae61-4729-b5a1-e904f90f7fcc-dbus\") pod \"global-pull-secret-syncer-q7hvp\" (UID: \"533b42cc-ae61-4729-b5a1-e904f90f7fcc\") " pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810599 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-run-netns\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810634 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.810969 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810670 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-var-lib-cni-multus\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810711 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vw55\" (UniqueName: \"kubernetes.io/projected/c799cb15-d6c2-4140-9af0-64b652841355-kube-api-access-9vw55\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810691 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-run-netns\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810730 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810732 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-sys\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810751 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-ovnkube-script-lib\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810768 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6df914a7-aff6-4e3a-936b-a8852546642f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810784 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/967e0e9f-301b-4f3c-ab27-df6f3cff68ee-iptables-alerter-script\") pod \"iptables-alerter-9v9sk\" (UID: \"967e0e9f-301b-4f3c-ab27-df6f3cff68ee\") " pod="openshift-network-operator/iptables-alerter-9v9sk" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810801 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp2nb\" (UniqueName: \"kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb\") pod \"network-check-target-9sfhz\" (UID: \"f311e199-3ddf-4650-8f2b-cde419d63d1a\") " pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810880 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-multus-socket-dir-parent\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810912 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-systemd-units\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810955 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-env-overrides\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.810978 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvlq4\" (UniqueName: \"kubernetes.io/projected/967e0e9f-301b-4f3c-ab27-df6f3cff68ee-kube-api-access-qvlq4\") pod \"iptables-alerter-9v9sk\" (UID: \"967e0e9f-301b-4f3c-ab27-df6f3cff68ee\") " pod="openshift-network-operator/iptables-alerter-9v9sk" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811000 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-multus-conf-dir\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811022 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c799cb15-d6c2-4140-9af0-64b652841355-multus-daemon-config\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811041 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-tuned\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811064 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6df914a7-aff6-4e3a-936b-a8852546642f-system-cni-dir\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.811783 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811085 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-etc-openvswitch\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811106 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-etc-selinux\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811131 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/533b42cc-ae61-4729-b5a1-e904f90f7fcc-kubelet-config\") pod \"global-pull-secret-syncer-q7hvp\" (UID: \"533b42cc-ae61-4729-b5a1-e904f90f7fcc\") " pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811152 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-device-dir\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811178 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56j8q\" (UniqueName: \"kubernetes.io/projected/9df22139-a68a-421b-807c-e6afb4bb691e-kube-api-access-56j8q\") pod \"node-ca-j5cfd\" (UID: \"9df22139-a68a-421b-807c-e6afb4bb691e\") " pod="openshift-image-registry/node-ca-j5cfd" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811207 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-var-lib-kubelet\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811210 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6df914a7-aff6-4e3a-936b-a8852546642f-system-cni-dir\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811228 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-host\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811220 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6df914a7-aff6-4e3a-936b-a8852546642f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811256 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3f4a0f2f-93e0-40f6-9ec7-6b61a09ade21-konnectivity-ca\") pod \"konnectivity-agent-tqkzp\" (UID: \"3f4a0f2f-93e0-40f6-9ec7-6b61a09ade21\") " pod="kube-system/konnectivity-agent-tqkzp" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811280 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-systemd-units\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811280 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-env-overrides\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811304 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-cni-netd\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811325 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-device-dir\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811359 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-cni-netd\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811394 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-etc-openvswitch\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811442 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-etc-selinux\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.812671 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811476 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wlmg\" (UniqueName: \"kubernetes.io/projected/6df914a7-aff6-4e3a-936b-a8852546642f-kube-api-access-8wlmg\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811500 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-run-ovn\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811522 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9df22139-a68a-421b-807c-e6afb4bb691e-host\") pod \"node-ca-j5cfd\" (UID: \"9df22139-a68a-421b-807c-e6afb4bb691e\") " pod="openshift-image-registry/node-ca-j5cfd" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811548 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-kubernetes\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811556 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-run-ovn\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811574 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-run-openvswitch\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811600 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9df22139-a68a-421b-807c-e6afb4bb691e-host\") pod \"node-ca-j5cfd\" (UID: \"9df22139-a68a-421b-807c-e6afb4bb691e\") " pod="openshift-image-registry/node-ca-j5cfd" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811600 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-socket-dir\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811637 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-sysctl-d\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811651 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-cnibin\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811666 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-run-openvswitch\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811676 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6df914a7-aff6-4e3a-936b-a8852546642f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811698 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-socket-dir\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811713 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6df914a7-aff6-4e3a-936b-a8852546642f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811750 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-log-socket\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811773 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-registration-dir\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811821 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-multus-cni-dir\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.813351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811842 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-etc-kubernetes\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.813875 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811861 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-modprobe-d\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.813875 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811853 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6df914a7-aff6-4e3a-936b-a8852546642f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.813875 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811886 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-slash\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.813875 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811901 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3f4a0f2f-93e0-40f6-9ec7-6b61a09ade21-konnectivity-ca\") pod \"konnectivity-agent-tqkzp\" (UID: \"3f4a0f2f-93e0-40f6-9ec7-6b61a09ade21\") " pod="kube-system/konnectivity-agent-tqkzp" Jul 24 13:42:48.813875 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811903 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-log-socket\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.813875 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811957 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/043101ce-c15f-4359-95d1-0432fd3498b6-registration-dir\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.813875 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811965 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-var-lib-openvswitch\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.813875 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811976 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-host-slash\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.813875 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.811907 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-var-lib-openvswitch\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.813875 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.812007 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-ovn-node-metrics-cert\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.813875 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.812031 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-hostroot\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.813875 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.812037 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6df914a7-aff6-4e3a-936b-a8852546642f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.813875 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.813842 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3f4a0f2f-93e0-40f6-9ec7-6b61a09ade21-agent-certs\") pod \"konnectivity-agent-tqkzp\" (UID: \"3f4a0f2f-93e0-40f6-9ec7-6b61a09ade21\") " pod="kube-system/konnectivity-agent-tqkzp" Jul 24 13:42:48.814410 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.814189 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-ovn-node-metrics-cert\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.816696 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.816670 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdk5n\" (UniqueName: \"kubernetes.io/projected/46030f6f-52fc-40f7-b5d4-7febe8bd998a-kube-api-access-xdk5n\") pod \"network-metrics-daemon-vjcfb\" (UID: \"46030f6f-52fc-40f7-b5d4-7febe8bd998a\") " pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:42:48.816896 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:48.816882 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:42:48.816958 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:48.816901 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:42:48.816958 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:48.816913 2557 projected.go:196] Error preparing data for projected volume kube-api-access-kp2nb for pod openshift-network-diagnostics/network-check-target-9sfhz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:42:48.817055 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:48.816977 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb podName:f311e199-3ddf-4650-8f2b-cde419d63d1a nodeName:}" failed. No retries permitted until 2026-07-24 13:42:49.316946988 +0000 UTC m=+3.003708417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kp2nb" (UniqueName: "kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb") pod "network-check-target-9sfhz" (UID: "f311e199-3ddf-4650-8f2b-cde419d63d1a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:42:48.817576 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.817220 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9jwj\" (UniqueName: \"kubernetes.io/projected/5513d77f-a9df-4125-bd63-d8f9df7b7e93-kube-api-access-w9jwj\") pod \"node-resolver-5mnw5\" (UID: \"5513d77f-a9df-4125-bd63-d8f9df7b7e93\") " pod="openshift-dns/node-resolver-5mnw5" Jul 24 13:42:48.817745 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.817724 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcsxg\" (UniqueName: \"kubernetes.io/projected/043101ce-c15f-4359-95d1-0432fd3498b6-kube-api-access-tcsxg\") pod \"aws-ebs-csi-driver-node-pzcrf\" (UID: \"043101ce-c15f-4359-95d1-0432fd3498b6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:48.819286 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.819261 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhssd\" (UniqueName: \"kubernetes.io/projected/27d03bcc-e0b1-4119-9b67-ca8c1cad55c3-kube-api-access-nhssd\") pod \"ovnkube-node-h8gk4\" (UID: \"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3\") " pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:48.820857 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.820177 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvlq4\" (UniqueName: \"kubernetes.io/projected/967e0e9f-301b-4f3c-ab27-df6f3cff68ee-kube-api-access-qvlq4\") pod \"iptables-alerter-9v9sk\" (UID: \"967e0e9f-301b-4f3c-ab27-df6f3cff68ee\") " pod="openshift-network-operator/iptables-alerter-9v9sk" Jul 24 13:42:48.820857 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.820358 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wlmg\" (UniqueName: \"kubernetes.io/projected/6df914a7-aff6-4e3a-936b-a8852546642f-kube-api-access-8wlmg\") pod \"multus-additional-cni-plugins-lmwnz\" (UID: \"6df914a7-aff6-4e3a-936b-a8852546642f\") " pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:48.823073 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.823051 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56j8q\" (UniqueName: \"kubernetes.io/projected/9df22139-a68a-421b-807c-e6afb4bb691e-kube-api-access-56j8q\") pod \"node-ca-j5cfd\" (UID: \"9df22139-a68a-421b-807c-e6afb4bb691e\") " pod="openshift-image-registry/node-ca-j5cfd" Jul 24 13:42:48.913099 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913066 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-multus-conf-dir\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.913099 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913104 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c799cb15-d6c2-4140-9af0-64b652841355-multus-daemon-config\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.913306 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913125 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-tuned\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.913306 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913142 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/533b42cc-ae61-4729-b5a1-e904f90f7fcc-kubelet-config\") pod \"global-pull-secret-syncer-q7hvp\" (UID: \"533b42cc-ae61-4729-b5a1-e904f90f7fcc\") " pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:48.913306 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913164 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-var-lib-kubelet\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.913306 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913184 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-host\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.913306 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913209 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-kubernetes\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.913306 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913230 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-sysctl-d\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.913306 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913250 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-cnibin\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.913306 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913274 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-multus-cni-dir\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.913306 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913290 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-etc-kubernetes\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.913306 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913307 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-modprobe-d\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.913639 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913333 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-hostroot\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.913639 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913352 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-systemd\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.913639 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913372 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret\") pod \"global-pull-secret-syncer-q7hvp\" (UID: \"533b42cc-ae61-4729-b5a1-e904f90f7fcc\") " pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:48.913639 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913387 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-sysctl-conf\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.913639 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913407 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-var-lib-kubelet\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.913639 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913430 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-system-cni-dir\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.913639 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913447 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-run-multus-certs\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.913639 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913468 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10d661f4-051c-4c4d-b80b-bb75aff5161d-tmp\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.913639 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913489 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-run-netns\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.913639 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913507 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-run\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.913639 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913528 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvbtr\" (UniqueName: \"kubernetes.io/projected/10d661f4-051c-4c4d-b80b-bb75aff5161d-kube-api-access-vvbtr\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.913639 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913567 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c799cb15-d6c2-4140-9af0-64b652841355-cni-binary-copy\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.913639 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913587 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-var-lib-cni-bin\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.913639 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913604 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-os-release\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.913639 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913631 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-run-k8s-cni-cncf-io\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.914158 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913662 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-sysconfig\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.914158 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913686 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-lib-modules\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.914158 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913710 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/533b42cc-ae61-4729-b5a1-e904f90f7fcc-dbus\") pod \"global-pull-secret-syncer-q7hvp\" (UID: \"533b42cc-ae61-4729-b5a1-e904f90f7fcc\") " pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:48.914158 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913739 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-var-lib-cni-multus\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.914158 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913753 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vw55\" (UniqueName: \"kubernetes.io/projected/c799cb15-d6c2-4140-9af0-64b652841355-kube-api-access-9vw55\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.914158 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913779 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-sys\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.914158 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913831 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-multus-socket-dir-parent\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.914158 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913919 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-multus-socket-dir-parent\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.914158 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913957 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-var-lib-kubelet\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.914158 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913971 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-system-cni-dir\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.914158 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.913998 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-multus-conf-dir\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.914158 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.914006 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-run-multus-certs\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.914158 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.914070 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-run-netns\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.914158 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.914105 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-run\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.914926 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.914900 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c799cb15-d6c2-4140-9af0-64b652841355-cni-binary-copy\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.915023 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.914971 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-var-lib-cni-bin\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.915023 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.915019 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-os-release\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.915125 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.915049 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-run-k8s-cni-cncf-io\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.915125 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.915084 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-sysconfig\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.915207 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.915177 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-lib-modules\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.915282 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.915268 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/533b42cc-ae61-4729-b5a1-e904f90f7fcc-dbus\") pod \"global-pull-secret-syncer-q7hvp\" (UID: \"533b42cc-ae61-4729-b5a1-e904f90f7fcc\") " pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:48.915348 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.915304 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-var-lib-cni-multus\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.915547 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.915522 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/533b42cc-ae61-4729-b5a1-e904f90f7fcc-kubelet-config\") pod \"global-pull-secret-syncer-q7hvp\" (UID: \"533b42cc-ae61-4729-b5a1-e904f90f7fcc\") " pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:48.915624 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.915556 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-host-var-lib-kubelet\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.915624 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.915600 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-host\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.915714 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.915644 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-kubernetes\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.916383 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.915753 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-sysctl-d\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.916383 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.915826 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-cnibin\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.916383 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.915869 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-multus-cni-dir\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.916383 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:48.915934 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:42:48.916383 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.915964 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c799cb15-d6c2-4140-9af0-64b652841355-multus-daemon-config\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.916383 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:48.915984 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret podName:533b42cc-ae61-4729-b5a1-e904f90f7fcc nodeName:}" failed. No retries permitted until 2026-07-24 13:42:49.415967195 +0000 UTC m=+3.102728589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret") pod "global-pull-secret-syncer-q7hvp" (UID: "533b42cc-ae61-4729-b5a1-e904f90f7fcc") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:42:48.916383 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.916027 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-etc-kubernetes\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.916383 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.916151 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-sysctl-conf\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.916383 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.916186 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-modprobe-d\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.916383 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.916217 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c799cb15-d6c2-4140-9af0-64b652841355-hostroot\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:48.916383 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.916241 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-systemd\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.916383 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.915531 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/10d661f4-051c-4c4d-b80b-bb75aff5161d-sys\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.916778 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.916483 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/10d661f4-051c-4c4d-b80b-bb75aff5161d-etc-tuned\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.917908 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.917890 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:42:48.918243 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.918226 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10d661f4-051c-4c4d-b80b-bb75aff5161d-tmp\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.921830 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.921784 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvbtr\" (UniqueName: \"kubernetes.io/projected/10d661f4-051c-4c4d-b80b-bb75aff5161d-kube-api-access-vvbtr\") pod \"tuned-p9gvn\" (UID: \"10d661f4-051c-4c4d-b80b-bb75aff5161d\") " pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:48.922290 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:48.922268 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vw55\" (UniqueName: \"kubernetes.io/projected/c799cb15-d6c2-4140-9af0-64b652841355-kube-api-access-9vw55\") pod \"multus-fdrqb\" (UID: \"c799cb15-d6c2-4140-9af0-64b652841355\") " pod="openshift-multus/multus-fdrqb" Jul 24 13:42:49.008324 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.008249 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j5cfd" Jul 24 13:42:49.014028 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.013994 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:42:49.021095 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.021070 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9v9sk" Jul 24 13:42:49.026512 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.026493 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tqkzp" Jul 24 13:42:49.032195 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.032165 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lmwnz" Jul 24 13:42:49.038380 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.038350 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" Jul 24 13:42:49.042889 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.042860 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5mnw5" Jul 24 13:42:49.049077 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.049060 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fdrqb" Jul 24 13:42:49.053211 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.053194 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" Jul 24 13:42:49.315652 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.315583 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs\") pod \"network-metrics-daemon-vjcfb\" (UID: \"46030f6f-52fc-40f7-b5d4-7febe8bd998a\") " pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:42:49.315786 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:49.315733 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:42:49.315861 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:49.315815 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs podName:46030f6f-52fc-40f7-b5d4-7febe8bd998a nodeName:}" failed. No retries permitted until 2026-07-24 13:42:50.315781703 +0000 UTC m=+4.002543097 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs") pod "network-metrics-daemon-vjcfb" (UID: "46030f6f-52fc-40f7-b5d4-7febe8bd998a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:42:49.416355 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.416320 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp2nb\" (UniqueName: \"kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb\") pod \"network-check-target-9sfhz\" (UID: \"f311e199-3ddf-4650-8f2b-cde419d63d1a\") " pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:42:49.416530 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.416376 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret\") pod \"global-pull-secret-syncer-q7hvp\" (UID: \"533b42cc-ae61-4729-b5a1-e904f90f7fcc\") " pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:49.416530 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:49.416477 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:42:49.416530 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:49.416503 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:42:49.416530 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:49.416531 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:42:49.416714 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:49.416539 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret podName:533b42cc-ae61-4729-b5a1-e904f90f7fcc nodeName:}" failed. No retries permitted until 2026-07-24 13:42:50.416523894 +0000 UTC m=+4.103285302 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret") pod "global-pull-secret-syncer-q7hvp" (UID: "533b42cc-ae61-4729-b5a1-e904f90f7fcc") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:42:49.416714 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:49.416544 2557 projected.go:196] Error preparing data for projected volume kube-api-access-kp2nb for pod openshift-network-diagnostics/network-check-target-9sfhz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:42:49.416714 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:49.416592 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb podName:f311e199-3ddf-4650-8f2b-cde419d63d1a nodeName:}" failed. No retries permitted until 2026-07-24 13:42:50.416576822 +0000 UTC m=+4.103338216 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kp2nb" (UniqueName: "kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb") pod "network-check-target-9sfhz" (UID: "f311e199-3ddf-4650-8f2b-cde419d63d1a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:42:49.604864 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:49.604844 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5513d77f_a9df_4125_bd63_d8f9df7b7e93.slice/crio-028a4d8e26e8bdcf9f3c109af11b3d72d1f52e736d8020d2a4db7be3a2f62026 WatchSource:0}: Error finding container 028a4d8e26e8bdcf9f3c109af11b3d72d1f52e736d8020d2a4db7be3a2f62026: Status 404 returned error can't find the container with id 028a4d8e26e8bdcf9f3c109af11b3d72d1f52e736d8020d2a4db7be3a2f62026 Jul 24 13:42:49.606027 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:49.606005 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df914a7_aff6_4e3a_936b_a8852546642f.slice/crio-80cb17a689aae0e1145083c71d9a7252a0bccb3d69cca8d2b364ceea50831773 WatchSource:0}: Error finding container 80cb17a689aae0e1145083c71d9a7252a0bccb3d69cca8d2b364ceea50831773: Status 404 returned error can't find the container with id 80cb17a689aae0e1145083c71d9a7252a0bccb3d69cca8d2b364ceea50831773 Jul 24 13:42:49.606662 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:49.606627 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod967e0e9f_301b_4f3c_ab27_df6f3cff68ee.slice/crio-3a137b85d5cf7c1137feeae44cf1ff1073250c0be7b32c45d737973fe7196d02 WatchSource:0}: Error finding container 3a137b85d5cf7c1137feeae44cf1ff1073250c0be7b32c45d737973fe7196d02: Status 404 returned error can't find the container with id 3a137b85d5cf7c1137feeae44cf1ff1073250c0be7b32c45d737973fe7196d02 Jul 24 13:42:49.607422 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:49.607348 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df22139_a68a_421b_807c_e6afb4bb691e.slice/crio-8d03a3c046a01a5b8c8c62050d4e62dadb30eadc3e11054abd0e1f430020abeb WatchSource:0}: Error finding container 8d03a3c046a01a5b8c8c62050d4e62dadb30eadc3e11054abd0e1f430020abeb: Status 404 returned error can't find the container with id 8d03a3c046a01a5b8c8c62050d4e62dadb30eadc3e11054abd0e1f430020abeb Jul 24 13:42:49.608081 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:49.608047 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod043101ce_c15f_4359_95d1_0432fd3498b6.slice/crio-e0dc12d4937f013e29df43fc80a2b47a43719a3fe4863ab1ffe396820566ffce WatchSource:0}: Error finding container e0dc12d4937f013e29df43fc80a2b47a43719a3fe4863ab1ffe396820566ffce: Status 404 returned error can't find the container with id e0dc12d4937f013e29df43fc80a2b47a43719a3fe4863ab1ffe396820566ffce Jul 24 13:42:49.614492 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:49.614460 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc799cb15_d6c2_4140_9af0_64b652841355.slice/crio-3a18110817c4b66b2ff86b670aa8b78d52883fca8c94632c741b393cf48dc288 WatchSource:0}: Error finding container 3a18110817c4b66b2ff86b670aa8b78d52883fca8c94632c741b393cf48dc288: Status 404 returned error can't find the container with id 3a18110817c4b66b2ff86b670aa8b78d52883fca8c94632c741b393cf48dc288 Jul 24 13:42:49.615245 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:42:49.615225 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d03bcc_e0b1_4119_9b67_ca8c1cad55c3.slice/crio-926c3a2cbd6827a221a4d3cd5fcbf2720b9f53f75fd4a4493ced7361a6dc223b WatchSource:0}: Error finding container 926c3a2cbd6827a221a4d3cd5fcbf2720b9f53f75fd4a4493ced7361a6dc223b: Status 404 returned error can't find the container with id 926c3a2cbd6827a221a4d3cd5fcbf2720b9f53f75fd4a4493ced7361a6dc223b Jul 24 13:42:49.734738 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.734555 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 13:37:48 +0000 UTC" deadline="2028-02-17 15:53:52.516273894 +0000 UTC" Jul 24 13:42:49.734738 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.734734 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13754h11m2.781543926s" Jul 24 13:42:49.863854 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.863744 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:42:49.871776 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.871755 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:42:49.871920 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:49.871899 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:42:49.883348 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.883321 2557 generic.go:356] "Generic (PLEG): container finished" podID="f08d9cec52ff7c0389f134ed4757b337" containerID="a7206e7a7847e240ce4689d073fd2461e4da5166614c167e0f69b484da2828e8" exitCode=0 Jul 24 13:42:49.883495 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.883391 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal" event={"ID":"f08d9cec52ff7c0389f134ed4757b337","Type":"ContainerDied","Data":"a7206e7a7847e240ce4689d073fd2461e4da5166614c167e0f69b484da2828e8"} Jul 24 13:42:49.884741 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.884712 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-250.ec2.internal" event={"ID":"c126e9785f9c1e44eb018143a4d59ea2","Type":"ContainerStarted","Data":"9ff3df4853281905809ca718c44e686aa69df6b409313a62f3dda7b294c47f8f"} Jul 24 13:42:49.885770 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.885751 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tqkzp" event={"ID":"3f4a0f2f-93e0-40f6-9ec7-6b61a09ade21","Type":"ContainerStarted","Data":"20f96a053dd05823097df985b401813ca90c689b2a1ef41478b117957af4c4d2"} Jul 24 13:42:49.886723 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.886705 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" event={"ID":"10d661f4-051c-4c4d-b80b-bb75aff5161d","Type":"ContainerStarted","Data":"d4072e320b78bff7c534f6443b07d306c7936acc74a3e14353ef84d1d9120df8"} Jul 24 13:42:49.887728 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.887640 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmwnz" event={"ID":"6df914a7-aff6-4e3a-936b-a8852546642f","Type":"ContainerStarted","Data":"80cb17a689aae0e1145083c71d9a7252a0bccb3d69cca8d2b364ceea50831773"} Jul 24 13:42:49.888706 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.888685 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" event={"ID":"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3","Type":"ContainerStarted","Data":"926c3a2cbd6827a221a4d3cd5fcbf2720b9f53f75fd4a4493ced7361a6dc223b"} Jul 24 13:42:49.889762 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.889745 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fdrqb" event={"ID":"c799cb15-d6c2-4140-9af0-64b652841355","Type":"ContainerStarted","Data":"3a18110817c4b66b2ff86b670aa8b78d52883fca8c94632c741b393cf48dc288"} Jul 24 13:42:49.890887 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.890844 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" event={"ID":"043101ce-c15f-4359-95d1-0432fd3498b6","Type":"ContainerStarted","Data":"e0dc12d4937f013e29df43fc80a2b47a43719a3fe4863ab1ffe396820566ffce"} Jul 24 13:42:49.892293 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.892277 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j5cfd" event={"ID":"9df22139-a68a-421b-807c-e6afb4bb691e","Type":"ContainerStarted","Data":"8d03a3c046a01a5b8c8c62050d4e62dadb30eadc3e11054abd0e1f430020abeb"} Jul 24 13:42:49.897726 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.897696 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9v9sk" event={"ID":"967e0e9f-301b-4f3c-ab27-df6f3cff68ee","Type":"ContainerStarted","Data":"3a137b85d5cf7c1137feeae44cf1ff1073250c0be7b32c45d737973fe7196d02"} Jul 24 13:42:49.899178 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:49.899156 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5mnw5" event={"ID":"5513d77f-a9df-4125-bd63-d8f9df7b7e93","Type":"ContainerStarted","Data":"028a4d8e26e8bdcf9f3c109af11b3d72d1f52e736d8020d2a4db7be3a2f62026"} Jul 24 13:42:50.321681 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:50.321642 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs\") pod \"network-metrics-daemon-vjcfb\" (UID: \"46030f6f-52fc-40f7-b5d4-7febe8bd998a\") " pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:42:50.321881 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:50.321832 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:42:50.321971 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:50.321900 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs podName:46030f6f-52fc-40f7-b5d4-7febe8bd998a nodeName:}" failed. No retries permitted until 2026-07-24 13:42:52.321878945 +0000 UTC m=+6.008640356 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs") pod "network-metrics-daemon-vjcfb" (UID: "46030f6f-52fc-40f7-b5d4-7febe8bd998a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:42:50.422379 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:50.422333 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp2nb\" (UniqueName: \"kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb\") pod \"network-check-target-9sfhz\" (UID: \"f311e199-3ddf-4650-8f2b-cde419d63d1a\") " pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:42:50.422545 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:50.422405 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret\") pod \"global-pull-secret-syncer-q7hvp\" (UID: \"533b42cc-ae61-4729-b5a1-e904f90f7fcc\") " pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:50.422545 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:50.422500 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:42:50.422545 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:50.422513 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:42:50.422545 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:50.422541 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:42:50.422756 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:50.422555 2557 projected.go:196] Error preparing data for projected volume kube-api-access-kp2nb for pod openshift-network-diagnostics/network-check-target-9sfhz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:42:50.422756 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:50.422564 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret podName:533b42cc-ae61-4729-b5a1-e904f90f7fcc nodeName:}" failed. No retries permitted until 2026-07-24 13:42:52.422547011 +0000 UTC m=+6.109308410 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret") pod "global-pull-secret-syncer-q7hvp" (UID: "533b42cc-ae61-4729-b5a1-e904f90f7fcc") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:42:50.422756 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:50.422610 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb podName:f311e199-3ddf-4650-8f2b-cde419d63d1a nodeName:}" failed. No retries permitted until 2026-07-24 13:42:52.422592347 +0000 UTC m=+6.109353746 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kp2nb" (UniqueName: "kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb") pod "network-check-target-9sfhz" (UID: "f311e199-3ddf-4650-8f2b-cde419d63d1a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:42:50.676560 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:50.676481 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:42:50.735365 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:50.735299 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 13:37:48 +0000 UTC" deadline="2028-01-20 09:29:55.527399054 +0000 UTC" Jul 24 13:42:50.735365 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:50.735362 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13075h47m4.792040882s" Jul 24 13:42:50.871530 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:50.871510 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:50.871703 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:50.871608 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:42:50.872089 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:50.872009 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:42:50.872161 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:50.872104 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:42:50.922197 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:50.922161 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal" event={"ID":"f08d9cec52ff7c0389f134ed4757b337","Type":"ContainerStarted","Data":"1ee91307159e4152e072dd62cdc38927f6387c9d558a4f60de017b3c9e69ec9e"} Jul 24 13:42:50.939105 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:50.939000 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-250.ec2.internal" podStartSLOduration=2.938980868 podStartE2EDuration="2.938980868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:42:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:42:49.911447705 +0000 UTC m=+3.598209118" watchObservedRunningTime="2026-07-24 13:42:50.938980868 +0000 UTC m=+4.625742286" Jul 24 13:42:51.872720 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:51.872174 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:42:51.872720 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:51.872331 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:42:52.338521 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:52.338426 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs\") pod \"network-metrics-daemon-vjcfb\" (UID: \"46030f6f-52fc-40f7-b5d4-7febe8bd998a\") " pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:42:52.338694 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:52.338620 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:42:52.338694 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:52.338682 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs podName:46030f6f-52fc-40f7-b5d4-7febe8bd998a nodeName:}" failed. No retries permitted until 2026-07-24 13:42:56.338664245 +0000 UTC m=+10.025425643 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs") pod "network-metrics-daemon-vjcfb" (UID: "46030f6f-52fc-40f7-b5d4-7febe8bd998a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:42:52.439209 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:52.439169 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp2nb\" (UniqueName: \"kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb\") pod \"network-check-target-9sfhz\" (UID: \"f311e199-3ddf-4650-8f2b-cde419d63d1a\") " pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:42:52.439390 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:52.439241 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret\") pod \"global-pull-secret-syncer-q7hvp\" (UID: \"533b42cc-ae61-4729-b5a1-e904f90f7fcc\") " pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:52.439390 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:52.439366 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:42:52.439501 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:52.439424 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret podName:533b42cc-ae61-4729-b5a1-e904f90f7fcc nodeName:}" failed. No retries permitted until 2026-07-24 13:42:56.439409918 +0000 UTC m=+10.126171318 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret") pod "global-pull-secret-syncer-q7hvp" (UID: "533b42cc-ae61-4729-b5a1-e904f90f7fcc") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:42:52.439847 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:52.439801 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:42:52.439847 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:52.439844 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:42:52.440007 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:52.439858 2557 projected.go:196] Error preparing data for projected volume kube-api-access-kp2nb for pod openshift-network-diagnostics/network-check-target-9sfhz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:42:52.440007 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:52.439903 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb podName:f311e199-3ddf-4650-8f2b-cde419d63d1a nodeName:}" failed. No retries permitted until 2026-07-24 13:42:56.439887324 +0000 UTC m=+10.126648724 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kp2nb" (UniqueName: "kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb") pod "network-check-target-9sfhz" (UID: "f311e199-3ddf-4650-8f2b-cde419d63d1a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:42:52.871935 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:52.871904 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:42:52.872126 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:52.872020 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:42:52.872467 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:52.872426 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:52.873460 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:52.873426 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:42:53.871562 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:53.871527 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:42:53.871739 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:53.871679 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:42:54.871986 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:54.871941 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:54.871986 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:54.871992 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:42:54.872528 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:54.872092 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:42:54.872628 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:54.872585 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:42:55.871905 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:55.871841 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:42:55.872103 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:55.871994 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:42:56.372675 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:56.372614 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs\") pod \"network-metrics-daemon-vjcfb\" (UID: \"46030f6f-52fc-40f7-b5d4-7febe8bd998a\") " pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:42:56.372866 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:56.372781 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:42:56.372866 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:56.372863 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs podName:46030f6f-52fc-40f7-b5d4-7febe8bd998a nodeName:}" failed. No retries permitted until 2026-07-24 13:43:04.37284369 +0000 UTC m=+18.059605091 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs") pod "network-metrics-daemon-vjcfb" (UID: "46030f6f-52fc-40f7-b5d4-7febe8bd998a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:42:56.473635 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:56.473594 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret\") pod \"global-pull-secret-syncer-q7hvp\" (UID: \"533b42cc-ae61-4729-b5a1-e904f90f7fcc\") " pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:56.473862 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:56.473670 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp2nb\" (UniqueName: \"kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb\") pod \"network-check-target-9sfhz\" (UID: \"f311e199-3ddf-4650-8f2b-cde419d63d1a\") " pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:42:56.473862 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:56.473761 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:42:56.473862 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:56.473801 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:42:56.473862 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:56.473840 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:42:56.473862 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:56.473851 2557 projected.go:196] Error preparing data for projected volume kube-api-access-kp2nb for pod openshift-network-diagnostics/network-check-target-9sfhz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:42:56.473862 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:56.473853 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret podName:533b42cc-ae61-4729-b5a1-e904f90f7fcc nodeName:}" failed. No retries permitted until 2026-07-24 13:43:04.473832496 +0000 UTC m=+18.160593905 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret") pod "global-pull-secret-syncer-q7hvp" (UID: "533b42cc-ae61-4729-b5a1-e904f90f7fcc") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:42:56.474155 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:56.473891 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb podName:f311e199-3ddf-4650-8f2b-cde419d63d1a nodeName:}" failed. No retries permitted until 2026-07-24 13:43:04.473878475 +0000 UTC m=+18.160639871 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kp2nb" (UniqueName: "kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb") pod "network-check-target-9sfhz" (UID: "f311e199-3ddf-4650-8f2b-cde419d63d1a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:42:56.873216 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:56.873181 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:42:56.873637 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:56.873302 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:42:56.873727 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:56.873699 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:56.873798 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:56.873784 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:42:57.871557 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:57.871524 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:42:57.871743 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:57.871649 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:42:58.871662 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:58.871624 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:42:58.872168 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:58.871776 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:42:58.872168 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:58.871792 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:42:58.872168 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:58.871900 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:42:59.871688 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:42:59.871656 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:42:59.872073 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:42:59.871765 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:43:00.871985 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:00.871954 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:00.872436 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:00.872060 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:43:00.872436 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:00.872091 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:00.872436 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:00.872139 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:43:01.872076 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:01.872041 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:01.872500 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:01.872182 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:43:02.872127 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:02.872093 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:02.872588 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:02.872092 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:02.872588 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:02.872233 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:43:02.872588 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:02.872278 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:43:03.872039 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:03.872008 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:03.872271 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:03.872138 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:43:04.437259 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:04.437214 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs\") pod \"network-metrics-daemon-vjcfb\" (UID: \"46030f6f-52fc-40f7-b5d4-7febe8bd998a\") " pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:04.437441 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:04.437378 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:43:04.437574 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:04.437459 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs podName:46030f6f-52fc-40f7-b5d4-7febe8bd998a nodeName:}" failed. No retries permitted until 2026-07-24 13:43:20.437436597 +0000 UTC m=+34.124197992 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs") pod "network-metrics-daemon-vjcfb" (UID: "46030f6f-52fc-40f7-b5d4-7febe8bd998a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:43:04.538236 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:04.538070 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp2nb\" (UniqueName: \"kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb\") pod \"network-check-target-9sfhz\" (UID: \"f311e199-3ddf-4650-8f2b-cde419d63d1a\") " pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:04.538236 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:04.538150 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret\") pod \"global-pull-secret-syncer-q7hvp\" (UID: \"533b42cc-ae61-4729-b5a1-e904f90f7fcc\") " pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:04.538453 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:04.538290 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:43:04.538453 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:04.538317 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:43:04.538453 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:04.538319 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:43:04.538453 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:04.538328 2557 projected.go:196] Error preparing data for projected volume kube-api-access-kp2nb for pod openshift-network-diagnostics/network-check-target-9sfhz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:43:04.538453 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:04.538381 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret podName:533b42cc-ae61-4729-b5a1-e904f90f7fcc nodeName:}" failed. No retries permitted until 2026-07-24 13:43:20.538365476 +0000 UTC m=+34.225126873 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret") pod "global-pull-secret-syncer-q7hvp" (UID: "533b42cc-ae61-4729-b5a1-e904f90f7fcc") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:43:04.538453 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:04.538401 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb podName:f311e199-3ddf-4650-8f2b-cde419d63d1a nodeName:}" failed. No retries permitted until 2026-07-24 13:43:20.538390307 +0000 UTC m=+34.225151701 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kp2nb" (UniqueName: "kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb") pod "network-check-target-9sfhz" (UID: "f311e199-3ddf-4650-8f2b-cde419d63d1a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:43:04.871687 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:04.871602 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:04.871687 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:04.871621 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:04.871915 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:04.871749 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:43:04.871915 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:04.871886 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:43:05.872106 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:05.872071 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:05.872543 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:05.872201 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:43:06.872747 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:06.872703 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:06.873229 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:06.872803 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:43:06.873229 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:06.872869 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:06.873229 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:06.872952 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:43:07.872410 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:07.872375 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:07.872577 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:07.872502 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:43:08.872352 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:08.872326 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:08.872669 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:08.872436 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:43:08.872669 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:08.872465 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:08.872669 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:08.872524 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:43:09.872214 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.871795 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:09.872427 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:09.872287 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:43:09.969515 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.969478 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tqkzp" event={"ID":"3f4a0f2f-93e0-40f6-9ec7-6b61a09ade21","Type":"ContainerStarted","Data":"74b5c565651ad04cb29222b1576fedf04c53fbb37edc4a57e40f5e4b1372534e"} Jul 24 13:43:09.971055 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.971025 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" event={"ID":"10d661f4-051c-4c4d-b80b-bb75aff5161d","Type":"ContainerStarted","Data":"263c92bcf73237811329e5e467e4d4b18a0d62e481ab730e33a723faf72e3817"} Jul 24 13:43:09.972712 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.972685 2557 generic.go:356] "Generic (PLEG): container finished" podID="6df914a7-aff6-4e3a-936b-a8852546642f" containerID="2e379ebc6b2992e35ad702af2081409192561e00000205cb78ecb8f759d135dc" exitCode=0 Jul 24 13:43:09.972855 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.972766 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmwnz" event={"ID":"6df914a7-aff6-4e3a-936b-a8852546642f","Type":"ContainerDied","Data":"2e379ebc6b2992e35ad702af2081409192561e00000205cb78ecb8f759d135dc"} Jul 24 13:43:09.976466 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.976441 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" event={"ID":"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3","Type":"ContainerStarted","Data":"2821bdba790e180eaa2d5129c7b4efc9ee62d6a804a8386763eacdbfad40b818"} Jul 24 13:43:09.976554 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.976508 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" event={"ID":"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3","Type":"ContainerStarted","Data":"7a869db68e6d396a4d04b519bddf683072b5d958b085d739c044d0d43fe129d6"} Jul 24 13:43:09.976554 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.976538 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" event={"ID":"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3","Type":"ContainerStarted","Data":"24eb42d745b504693a2e0cdbb44510232f72fbe578ebff65ba10af6b77a29c05"} Jul 24 13:43:09.976620 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.976565 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" event={"ID":"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3","Type":"ContainerStarted","Data":"71b6cc0f6d4cd9eecd291c424448e0409de132d7436038df8b7928663727f3ad"} Jul 24 13:43:09.976620 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.976592 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" event={"ID":"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3","Type":"ContainerStarted","Data":"dbd49def7df285c7f932e8df24ca1dc993c326dca2bc9c032b808a0a4ce8a98b"} Jul 24 13:43:09.976684 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.976620 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" event={"ID":"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3","Type":"ContainerStarted","Data":"cf66675625a67a8c91077bb42bb13fd2e4af80bb8f435b56328e0a4abd7ad239"} Jul 24 13:43:09.978481 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.978296 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fdrqb" event={"ID":"c799cb15-d6c2-4140-9af0-64b652841355","Type":"ContainerStarted","Data":"97e0091968c705519417041f3a1b2e78a197a6073a8b97d5124c82a192e4b7b4"} Jul 24 13:43:09.979986 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.979966 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" event={"ID":"043101ce-c15f-4359-95d1-0432fd3498b6","Type":"ContainerStarted","Data":"10391c9eb56930a4f513bc8c1bd36cfc35dc58dfe29ed78d7ce55e435e8cff3d"} Jul 24 13:43:09.981657 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.981621 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j5cfd" event={"ID":"9df22139-a68a-421b-807c-e6afb4bb691e","Type":"ContainerStarted","Data":"034889d138fd1d20d7a8982a51ad0de3d4e177a12f01e35fd3d78dbb21acbf27"} Jul 24 13:43:09.983132 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.983110 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5mnw5" event={"ID":"5513d77f-a9df-4125-bd63-d8f9df7b7e93","Type":"ContainerStarted","Data":"dc3b2b66d12b1b5dfbcd9ab5770986e9f2e29eede31924f57931301dacadf8a8"} Jul 24 13:43:09.984483 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.984449 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-250.ec2.internal" podStartSLOduration=21.984432822 podStartE2EDuration="21.984432822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:42:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:42:50.940352453 +0000 UTC m=+4.627113870" watchObservedRunningTime="2026-07-24 13:43:09.984432822 +0000 UTC m=+23.671194240" Jul 24 13:43:09.998435 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.998388 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tqkzp" podStartSLOduration=4.735217277 podStartE2EDuration="23.998374428s" totalImagesPullingTime="19.263157151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:43:09.984673764 +0000 UTC m=+23.671435181" watchObservedRunningTime="2026-07-24 13:43:09.998374428 +0000 UTC m=+23.685135844" Jul 24 13:43:09.998590 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:09.998465 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-j5cfd" podStartSLOduration=4.739934945 podStartE2EDuration="23.998462014s" totalImagesPullingTime="19.258527069s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:43:09.998235114 +0000 UTC m=+23.684996532" watchObservedRunningTime="2026-07-24 13:43:09.998462014 +0000 UTC m=+23.685223475" Jul 24 13:43:10.033823 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:10.033763 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fdrqb" podStartSLOduration=4.71445771 podStartE2EDuration="24.033749095s" totalImagesPullingTime="19.319291385s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:43:10.033469105 +0000 UTC m=+23.720230522" watchObservedRunningTime="2026-07-24 13:43:10.033749095 +0000 UTC m=+23.720510513" Jul 24 13:43:10.048193 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:10.048145 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5mnw5" podStartSLOduration=4.771439937 podStartE2EDuration="24.048131853s" totalImagesPullingTime="19.276691916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:43:10.047617914 +0000 UTC m=+23.734379333" watchObservedRunningTime="2026-07-24 13:43:10.048131853 +0000 UTC m=+23.734893269" Jul 24 13:43:10.063421 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:10.063376 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-p9gvn" podStartSLOduration=4.800780902 podStartE2EDuration="24.063362005s" totalImagesPullingTime="19.262581103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:43:10.06329471 +0000 UTC m=+23.750056129" watchObservedRunningTime="2026-07-24 13:43:10.063362005 +0000 UTC m=+23.750123422" Jul 24 13:43:10.308872 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:10.308845 2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 24 13:43:10.786625 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:10.786509 2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T13:43:10.308867871Z","UUID":"16d7800c-3a9a-47fd-991f-fd39caf4dca3","Handler":null,"Name":"","Endpoint":""} Jul 24 13:43:10.788403 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:10.788379 2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 24 13:43:10.788546 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:10.788421 2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 24 13:43:10.872081 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:10.872039 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:10.872081 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:10.872084 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:10.872312 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:10.872169 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:43:10.872418 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:10.872387 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:43:10.987632 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:10.987586 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" event={"ID":"043101ce-c15f-4359-95d1-0432fd3498b6","Type":"ContainerStarted","Data":"b50b11d009f2bccc7d52b0e6134fde644972f7274171462dbce6a41c968c887e"} Jul 24 13:43:10.989704 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:10.989675 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9v9sk" event={"ID":"967e0e9f-301b-4f3c-ab27-df6f3cff68ee","Type":"ContainerStarted","Data":"6291d6eef39d123f2eb7b6ce02e188e59cd7294e9a1a2d6b92120ebf5e3feb9b"} Jul 24 13:43:11.004692 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:11.004639 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9v9sk" podStartSLOduration=5.735214259 podStartE2EDuration="25.004616789s" totalImagesPullingTime="19.26940253s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:43:11.004008285 +0000 UTC m=+24.690769703" watchObservedRunningTime="2026-07-24 13:43:11.004616789 +0000 UTC m=+24.691378209" Jul 24 13:43:11.872100 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:11.872068 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:11.872288 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:11.872228 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:43:11.996563 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:11.996446 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" event={"ID":"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3","Type":"ContainerStarted","Data":"3fa5238ff95d1c45ce4b9211cca0c99d4e23c52200f9d69aac52de70a0b343f5"} Jul 24 13:43:11.998866 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:11.998841 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" event={"ID":"043101ce-c15f-4359-95d1-0432fd3498b6","Type":"ContainerStarted","Data":"6be15e6173e0cae4485d7df8d54f8891b7521ee11ced81fdf6f85432a97741a8"} Jul 24 13:43:12.014799 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:12.014748 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pzcrf" podStartSLOduration=4.48477034 podStartE2EDuration="26.014729038s" totalImagesPullingTime="21.529958698s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:42:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:43:12.013736261 +0000 UTC m=+25.700497678" watchObservedRunningTime="2026-07-24 13:43:12.014729038 +0000 UTC m=+25.701490502" Jul 24 13:43:12.198516 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:12.198479 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tqkzp" Jul 24 13:43:12.199146 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:12.199121 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tqkzp" Jul 24 13:43:12.872196 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:12.871994 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:12.872384 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:12.871999 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:12.872384 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:12.872311 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:43:12.872384 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:12.872359 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:43:13.001324 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:13.001265 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tqkzp" Jul 24 13:43:13.001919 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:13.001896 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tqkzp" Jul 24 13:43:13.871782 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:13.871751 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:13.872006 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:13.871926 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:43:14.871990 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:14.871725 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:14.872672 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:14.871787 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:14.872672 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:14.872102 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:43:14.872672 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:14.872153 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:43:15.006966 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:15.006933 2557 generic.go:356] "Generic (PLEG): container finished" podID="6df914a7-aff6-4e3a-936b-a8852546642f" containerID="3cce880b159083733ebf97d9c0092714ccba9db8c450a2bc7597b5d61b06eafa" exitCode=0 Jul 24 13:43:15.007137 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:15.007010 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmwnz" event={"ID":"6df914a7-aff6-4e3a-936b-a8852546642f","Type":"ContainerDied","Data":"3cce880b159083733ebf97d9c0092714ccba9db8c450a2bc7597b5d61b06eafa"} Jul 24 13:43:15.010900 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:15.010838 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" event={"ID":"27d03bcc-e0b1-4119-9b67-ca8c1cad55c3","Type":"ContainerStarted","Data":"e33d8b8f2ef9051e8417f09c2b137b9777e4b2132376556b867f2e88280cb85c"} Jul 24 13:43:15.011335 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:15.011287 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:43:15.011482 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:15.011373 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:43:15.011482 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:15.011417 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:43:15.027457 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:15.027432 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:43:15.027578 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:15.027550 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:43:15.052083 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:15.052038 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" podStartSLOduration=9.726047943 podStartE2EDuration="29.052021554s" totalImagesPullingTime="19.325973611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:43:15.051469065 +0000 UTC m=+28.738230473" watchObservedRunningTime="2026-07-24 13:43:15.052021554 +0000 UTC m=+28.738782971" Jul 24 13:43:15.871904 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:15.871864 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:15.872158 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:15.871983 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:43:16.731828 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:16.731786 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vjcfb"] Jul 24 13:43:16.732009 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:16.731916 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:16.732081 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:16.732043 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:43:16.735333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:16.735306 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9sfhz"] Jul 24 13:43:16.735478 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:16.735466 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:16.735595 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:16.735572 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:43:16.736116 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:16.736094 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q7hvp"] Jul 24 13:43:16.736230 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:16.736187 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:16.736305 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:16.736287 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:43:17.016576 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:17.016542 2557 generic.go:356] "Generic (PLEG): container finished" podID="6df914a7-aff6-4e3a-936b-a8852546642f" containerID="6bbecb4288102bbc753bf1b7ca74455388e814c1a6a7cec20ec0e963c857b087" exitCode=0 Jul 24 13:43:17.017083 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:17.016620 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmwnz" event={"ID":"6df914a7-aff6-4e3a-936b-a8852546642f","Type":"ContainerDied","Data":"6bbecb4288102bbc753bf1b7ca74455388e814c1a6a7cec20ec0e963c857b087"} Jul 24 13:43:18.872502 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:18.872323 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:18.872904 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:18.872323 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:18.872904 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:18.872585 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:43:18.872904 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:18.872323 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:18.872904 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:18.872659 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:43:18.872904 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:18.872736 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:43:19.024164 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:19.024129 2557 generic.go:356] "Generic (PLEG): container finished" podID="6df914a7-aff6-4e3a-936b-a8852546642f" containerID="5389f26698484fc85a95c045bb65d04018a0d1bbd9b63bce023e4fc52e86e7dd" exitCode=0 Jul 24 13:43:19.024363 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:19.024202 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmwnz" event={"ID":"6df914a7-aff6-4e3a-936b-a8852546642f","Type":"ContainerDied","Data":"5389f26698484fc85a95c045bb65d04018a0d1bbd9b63bce023e4fc52e86e7dd"} Jul 24 13:43:20.452819 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:20.452766 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs\") pod \"network-metrics-daemon-vjcfb\" (UID: \"46030f6f-52fc-40f7-b5d4-7febe8bd998a\") " pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:20.453383 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:20.452978 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:43:20.453383 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:20.453039 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs podName:46030f6f-52fc-40f7-b5d4-7febe8bd998a nodeName:}" failed. No retries permitted until 2026-07-24 13:43:52.453025979 +0000 UTC m=+66.139787374 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs") pod "network-metrics-daemon-vjcfb" (UID: "46030f6f-52fc-40f7-b5d4-7febe8bd998a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:43:20.553499 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:20.553458 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp2nb\" (UniqueName: \"kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb\") pod \"network-check-target-9sfhz\" (UID: \"f311e199-3ddf-4650-8f2b-cde419d63d1a\") " pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:20.553689 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:20.553527 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret\") pod \"global-pull-secret-syncer-q7hvp\" (UID: \"533b42cc-ae61-4729-b5a1-e904f90f7fcc\") " pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:20.553689 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:20.553636 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:43:20.553689 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:20.553658 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:43:20.553689 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:20.553656 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:43:20.553689 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:20.553673 2557 projected.go:196] Error preparing data for projected volume kube-api-access-kp2nb for pod openshift-network-diagnostics/network-check-target-9sfhz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:43:20.553987 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:20.553723 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret podName:533b42cc-ae61-4729-b5a1-e904f90f7fcc nodeName:}" failed. No retries permitted until 2026-07-24 13:43:52.553704236 +0000 UTC m=+66.240465654 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret") pod "global-pull-secret-syncer-q7hvp" (UID: "533b42cc-ae61-4729-b5a1-e904f90f7fcc") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:43:20.553987 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:20.553740 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb podName:f311e199-3ddf-4650-8f2b-cde419d63d1a nodeName:}" failed. No retries permitted until 2026-07-24 13:43:52.553733956 +0000 UTC m=+66.240495351 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kp2nb" (UniqueName: "kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb") pod "network-check-target-9sfhz" (UID: "f311e199-3ddf-4650-8f2b-cde419d63d1a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:43:20.872772 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:20.872153 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:20.872772 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:20.872167 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:20.872772 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:20.872363 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9sfhz" podUID="f311e199-3ddf-4650-8f2b-cde419d63d1a" Jul 24 13:43:20.872772 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:20.872438 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7hvp" podUID="533b42cc-ae61-4729-b5a1-e904f90f7fcc" Jul 24 13:43:20.872772 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:20.872487 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:20.872772 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:20.872562 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vjcfb" podUID="46030f6f-52fc-40f7-b5d4-7febe8bd998a" Jul 24 13:43:21.673424 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.673353 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-250.ec2.internal" event="NodeReady" Jul 24 13:43:21.673866 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.673493 2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 24 13:43:21.709656 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.709619 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-644fb596d7-kjvss"] Jul 24 13:43:21.714580 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.714524 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.716465 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.716432 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 24 13:43:21.717040 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.717008 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 24 13:43:21.717161 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.717087 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 24 13:43:21.717161 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.717147 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qzzs2\"" Jul 24 13:43:21.722310 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.722284 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-h2vp7"] Jul 24 13:43:21.726776 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.725653 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 24 13:43:21.727132 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.727112 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-25lb7"] Jul 24 13:43:21.727284 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.727262 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:21.729504 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.729480 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 24 13:43:21.729504 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.729496 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 24 13:43:21.729775 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.729758 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rdndh\"" Jul 24 13:43:21.731197 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.731174 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-644fb596d7-kjvss"] Jul 24 13:43:21.731306 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.731291 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-25lb7" Jul 24 13:43:21.733198 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.733184 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 24 13:43:21.733549 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.733531 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 24 13:43:21.733902 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.733801 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 24 13:43:21.733902 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.733869 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c9hpz\"" Jul 24 13:43:21.734753 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.734734 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-25lb7"] Jul 24 13:43:21.740484 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.740456 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h2vp7"] Jul 24 13:43:21.863270 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.863233 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-image-registry-private-configuration\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.863452 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.863278 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-ca-trust-extracted\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.863452 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.863364 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9e7565d1-c59f-4fd9-adcd-09042dd2345e-config-volume\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:21.863452 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.863406 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-bound-sa-token\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.863452 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.863432 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.863452 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.863450 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-installation-pull-secrets\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.863692 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.863480 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-trusted-ca\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.863692 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.863503 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rvpm\" (UniqueName: \"kubernetes.io/projected/9e7565d1-c59f-4fd9-adcd-09042dd2345e-kube-api-access-7rvpm\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:21.863692 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.863529 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkpbx\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-kube-api-access-xkpbx\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.863692 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.863624 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/849e0755-ade6-4136-9394-017410ace217-cert\") pod \"ingress-canary-25lb7\" (UID: \"849e0755-ade6-4136-9394-017410ace217\") " pod="openshift-ingress-canary/ingress-canary-25lb7" Jul 24 13:43:21.863692 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.863669 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-certificates\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.863956 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.863696 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p84nh\" (UniqueName: \"kubernetes.io/projected/849e0755-ade6-4136-9394-017410ace217-kube-api-access-p84nh\") pod \"ingress-canary-25lb7\" (UID: \"849e0755-ade6-4136-9394-017410ace217\") " pod="openshift-ingress-canary/ingress-canary-25lb7" Jul 24 13:43:21.863956 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.863725 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:21.863956 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.863747 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7565d1-c59f-4fd9-adcd-09042dd2345e-tmp-dir\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:21.964395 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.964319 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-ca-trust-extracted\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.964395 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.964367 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9e7565d1-c59f-4fd9-adcd-09042dd2345e-config-volume\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:21.964395 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.964390 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-bound-sa-token\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.964657 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.964414 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.964657 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.964438 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-installation-pull-secrets\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.964657 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.964465 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-trusted-ca\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.964657 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:21.964526 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:43:21.964657 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:21.964546 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644fb596d7-kjvss: secret "image-registry-tls" not found Jul 24 13:43:21.964657 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:21.964601 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls podName:fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73 nodeName:}" failed. No retries permitted until 2026-07-24 13:43:22.464579766 +0000 UTC m=+36.151341192 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls") pod "image-registry-644fb596d7-kjvss" (UID: "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73") : secret "image-registry-tls" not found Jul 24 13:43:21.965064 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.964710 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rvpm\" (UniqueName: \"kubernetes.io/projected/9e7565d1-c59f-4fd9-adcd-09042dd2345e-kube-api-access-7rvpm\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:21.965064 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.964749 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkpbx\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-kube-api-access-xkpbx\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.965064 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.964802 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/849e0755-ade6-4136-9394-017410ace217-cert\") pod \"ingress-canary-25lb7\" (UID: \"849e0755-ade6-4136-9394-017410ace217\") " pod="openshift-ingress-canary/ingress-canary-25lb7" Jul 24 13:43:21.965064 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.964868 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-certificates\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.965064 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.964896 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p84nh\" (UniqueName: \"kubernetes.io/projected/849e0755-ade6-4136-9394-017410ace217-kube-api-access-p84nh\") pod \"ingress-canary-25lb7\" (UID: \"849e0755-ade6-4136-9394-017410ace217\") " pod="openshift-ingress-canary/ingress-canary-25lb7" Jul 24 13:43:21.965064 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.964922 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:21.965064 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.964933 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-ca-trust-extracted\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.965064 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.964945 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7565d1-c59f-4fd9-adcd-09042dd2345e-tmp-dir\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:21.965064 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.965044 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9e7565d1-c59f-4fd9-adcd-09042dd2345e-config-volume\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:21.965064 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:21.965056 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:43:21.965556 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:21.965101 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls podName:9e7565d1-c59f-4fd9-adcd-09042dd2345e nodeName:}" failed. No retries permitted until 2026-07-24 13:43:22.465086138 +0000 UTC m=+36.151847552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls") pod "dns-default-h2vp7" (UID: "9e7565d1-c59f-4fd9-adcd-09042dd2345e") : secret "dns-default-metrics-tls" not found Jul 24 13:43:21.965556 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.965237 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-image-registry-private-configuration\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.965556 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.965347 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7565d1-c59f-4fd9-adcd-09042dd2345e-tmp-dir\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:21.965556 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.965417 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-trusted-ca\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.965556 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.965522 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-certificates\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.969053 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.969023 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-image-registry-private-configuration\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.969053 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.969041 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-installation-pull-secrets\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.969362 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.969347 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/849e0755-ade6-4136-9394-017410ace217-cert\") pod \"ingress-canary-25lb7\" (UID: \"849e0755-ade6-4136-9394-017410ace217\") " pod="openshift-ingress-canary/ingress-canary-25lb7" Jul 24 13:43:21.973585 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.973561 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rvpm\" (UniqueName: \"kubernetes.io/projected/9e7565d1-c59f-4fd9-adcd-09042dd2345e-kube-api-access-7rvpm\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:21.974120 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.974097 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p84nh\" (UniqueName: \"kubernetes.io/projected/849e0755-ade6-4136-9394-017410ace217-kube-api-access-p84nh\") pod \"ingress-canary-25lb7\" (UID: \"849e0755-ade6-4136-9394-017410ace217\") " pod="openshift-ingress-canary/ingress-canary-25lb7" Jul 24 13:43:21.974378 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.974362 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-bound-sa-token\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:21.974986 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:21.974966 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkpbx\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-kube-api-access-xkpbx\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:22.046533 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:22.046496 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-25lb7" Jul 24 13:43:22.469710 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:22.469676 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:22.469910 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:22.469772 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:22.469910 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:22.469860 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:43:22.470015 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:22.469922 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls podName:9e7565d1-c59f-4fd9-adcd-09042dd2345e nodeName:}" failed. No retries permitted until 2026-07-24 13:43:23.469905754 +0000 UTC m=+37.156667152 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls") pod "dns-default-h2vp7" (UID: "9e7565d1-c59f-4fd9-adcd-09042dd2345e") : secret "dns-default-metrics-tls" not found Jul 24 13:43:22.470015 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:22.469961 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:43:22.470015 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:22.469983 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644fb596d7-kjvss: secret "image-registry-tls" not found Jul 24 13:43:22.470142 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:22.470051 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls podName:fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73 nodeName:}" failed. No retries permitted until 2026-07-24 13:43:23.470035413 +0000 UTC m=+37.156796825 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls") pod "image-registry-644fb596d7-kjvss" (UID: "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73") : secret "image-registry-tls" not found Jul 24 13:43:22.872233 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:22.872140 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:22.872916 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:22.872140 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:22.872916 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:22.872153 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:22.875360 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:22.875155 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 13:43:22.875360 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:22.875190 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 13:43:22.875360 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:22.875234 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 13:43:22.875360 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:22.875191 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 13:43:22.875360 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:22.875155 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cbzbq\"" Jul 24 13:43:22.875696 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:22.875527 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l5jkr\"" Jul 24 13:43:23.233374 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:23.233163 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-25lb7"] Jul 24 13:43:23.236469 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:43:23.236279 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod849e0755_ade6_4136_9394_017410ace217.slice/crio-fd60ea82d72b1831c732c6e073291c2703944bddc376f3ac8112a93ad4e5787b WatchSource:0}: Error finding container fd60ea82d72b1831c732c6e073291c2703944bddc376f3ac8112a93ad4e5787b: Status 404 returned error can't find the container with id fd60ea82d72b1831c732c6e073291c2703944bddc376f3ac8112a93ad4e5787b Jul 24 13:43:23.479131 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:23.479021 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:23.479131 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:23.479087 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:23.479301 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:23.479200 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:43:23.479301 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:23.479202 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:43:23.479301 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:23.479265 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls podName:9e7565d1-c59f-4fd9-adcd-09042dd2345e nodeName:}" failed. No retries permitted until 2026-07-24 13:43:25.479248969 +0000 UTC m=+39.166010366 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls") pod "dns-default-h2vp7" (UID: "9e7565d1-c59f-4fd9-adcd-09042dd2345e") : secret "dns-default-metrics-tls" not found Jul 24 13:43:23.479301 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:23.479211 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644fb596d7-kjvss: secret "image-registry-tls" not found Jul 24 13:43:23.479441 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:23.479325 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls podName:fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73 nodeName:}" failed. No retries permitted until 2026-07-24 13:43:25.479313879 +0000 UTC m=+39.166075274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls") pod "image-registry-644fb596d7-kjvss" (UID: "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73") : secret "image-registry-tls" not found Jul 24 13:43:24.038988 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:24.038951 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-25lb7" event={"ID":"849e0755-ade6-4136-9394-017410ace217","Type":"ContainerStarted","Data":"fd60ea82d72b1831c732c6e073291c2703944bddc376f3ac8112a93ad4e5787b"} Jul 24 13:43:24.042409 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:24.042382 2557 generic.go:356] "Generic (PLEG): container finished" podID="6df914a7-aff6-4e3a-936b-a8852546642f" containerID="74ec501b07e119eedbde545bb5312a31f8b29b2eb477857639146d2592a6fe8b" exitCode=0 Jul 24 13:43:24.042542 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:24.042444 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmwnz" event={"ID":"6df914a7-aff6-4e3a-936b-a8852546642f","Type":"ContainerDied","Data":"74ec501b07e119eedbde545bb5312a31f8b29b2eb477857639146d2592a6fe8b"} Jul 24 13:43:25.049130 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:25.049091 2557 generic.go:356] "Generic (PLEG): container finished" podID="6df914a7-aff6-4e3a-936b-a8852546642f" containerID="7ef71d71172d37e2ce6028b19d6ce54d53544c5f1a3a58e9f303c491e964d0b3" exitCode=0 Jul 24 13:43:25.049559 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:25.049140 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmwnz" event={"ID":"6df914a7-aff6-4e3a-936b-a8852546642f","Type":"ContainerDied","Data":"7ef71d71172d37e2ce6028b19d6ce54d53544c5f1a3a58e9f303c491e964d0b3"} Jul 24 13:43:25.497232 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:25.497200 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:25.497373 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:25.497279 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:25.497373 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:25.497353 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:43:25.497373 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:25.497353 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:43:25.497373 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:25.497367 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644fb596d7-kjvss: secret "image-registry-tls" not found Jul 24 13:43:25.497509 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:25.497416 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls podName:fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73 nodeName:}" failed. No retries permitted until 2026-07-24 13:43:29.497397078 +0000 UTC m=+43.184158491 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls") pod "image-registry-644fb596d7-kjvss" (UID: "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73") : secret "image-registry-tls" not found Jul 24 13:43:25.497509 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:25.497431 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls podName:9e7565d1-c59f-4fd9-adcd-09042dd2345e nodeName:}" failed. No retries permitted until 2026-07-24 13:43:29.497423865 +0000 UTC m=+43.184185264 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls") pod "dns-default-h2vp7" (UID: "9e7565d1-c59f-4fd9-adcd-09042dd2345e") : secret "dns-default-metrics-tls" not found Jul 24 13:43:26.053126 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.052845 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-25lb7" event={"ID":"849e0755-ade6-4136-9394-017410ace217","Type":"ContainerStarted","Data":"3c033b855390a48c258bab9be3dea19d5c2f5311ca80624f1c19b04befc8cb04"} Jul 24 13:43:26.056112 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.056084 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lmwnz" event={"ID":"6df914a7-aff6-4e3a-936b-a8852546642f","Type":"ContainerStarted","Data":"d2c52d528a0132fc5b7bdb3882b400239f3451558f6efc4d2203b9c52df205d0"} Jul 24 13:43:26.068970 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.068920 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-25lb7" podStartSLOduration=2.744628393 podStartE2EDuration="5.068901697s" totalImagesPullingTime="2.324273304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:43:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:43:26.068417688 +0000 UTC m=+39.755179104" watchObservedRunningTime="2026-07-24 13:43:26.068901697 +0000 UTC m=+39.755663115" Jul 24 13:43:26.088929 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.088873 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lmwnz" podStartSLOduration=10.087189692 podStartE2EDuration="40.08885555s" totalImagesPullingTime="30.001665858s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:42:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:43:26.087922888 +0000 UTC m=+39.774684305" watchObservedRunningTime="2026-07-24 13:43:26.08885555 +0000 UTC m=+39.775616964" Jul 24 13:43:26.481919 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.481883 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fvvgk"] Jul 24 13:43:26.498924 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.498893 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fvvgk"] Jul 24 13:43:26.499065 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.499014 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:26.502526 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.502502 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 24 13:43:26.502526 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.502517 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 24 13:43:26.502894 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.502880 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n5ltr\"" Jul 24 13:43:26.502894 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.502887 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 24 13:43:26.503012 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.502989 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 24 13:43:26.606073 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.605999 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:26.606238 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.606113 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d96004d-7277-4a7e-9ec7-a87faa9526fd-crio-socket\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:26.606238 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.606149 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d96004d-7277-4a7e-9ec7-a87faa9526fd-data-volume\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:26.606238 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.606174 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-278xp\" (UniqueName: \"kubernetes.io/projected/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-api-access-278xp\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:26.606238 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.606215 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:26.613036 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.613010 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-brq2q"] Jul 24 13:43:26.634669 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.634636 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-brq2q"] Jul 24 13:43:26.634792 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.634738 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-brq2q" Jul 24 13:43:26.636934 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.636913 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 24 13:43:26.637034 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.636992 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-rs9pf\"" Jul 24 13:43:26.637034 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.637022 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 24 13:43:26.706597 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.706563 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d96004d-7277-4a7e-9ec7-a87faa9526fd-crio-socket\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:26.706597 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.706600 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d96004d-7277-4a7e-9ec7-a87faa9526fd-data-volume\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:26.706867 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.706619 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-278xp\" (UniqueName: \"kubernetes.io/projected/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-api-access-278xp\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:26.706867 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.706836 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:26.706981 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.706869 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d4h4\" (UniqueName: \"kubernetes.io/projected/0e3799ec-fba5-4ec2-b848-495b192761e8-kube-api-access-7d4h4\") pod \"migrator-6865c9967-brq2q\" (UID: \"0e3799ec-fba5-4ec2-b848-495b192761e8\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-brq2q" Jul 24 13:43:26.706981 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:26.706888 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 13:43:26.706981 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.706838 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d96004d-7277-4a7e-9ec7-a87faa9526fd-crio-socket\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:26.706981 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.706949 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:26.706981 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:26.706971 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-rbac-proxy-cm podName:3d96004d-7277-4a7e-9ec7-a87faa9526fd nodeName:}" failed. No retries permitted until 2026-07-24 13:43:27.206950463 +0000 UTC m=+40.893711858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fvvgk" (UID: "3d96004d-7277-4a7e-9ec7-a87faa9526fd") : configmap "kube-rbac-proxy" not found Jul 24 13:43:26.707192 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:26.707024 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 13:43:26.707192 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:26.707068 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls podName:3d96004d-7277-4a7e-9ec7-a87faa9526fd nodeName:}" failed. No retries permitted until 2026-07-24 13:43:27.207056334 +0000 UTC m=+40.893817749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fvvgk" (UID: "3d96004d-7277-4a7e-9ec7-a87faa9526fd") : secret "insights-runtime-extractor-tls" not found Jul 24 13:43:26.707192 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.707077 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d96004d-7277-4a7e-9ec7-a87faa9526fd-data-volume\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:26.720299 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.720261 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-278xp\" (UniqueName: \"kubernetes.io/projected/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-api-access-278xp\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:26.807457 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.807377 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d4h4\" (UniqueName: \"kubernetes.io/projected/0e3799ec-fba5-4ec2-b848-495b192761e8-kube-api-access-7d4h4\") pod \"migrator-6865c9967-brq2q\" (UID: \"0e3799ec-fba5-4ec2-b848-495b192761e8\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-brq2q" Jul 24 13:43:26.815916 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.815886 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d4h4\" (UniqueName: \"kubernetes.io/projected/0e3799ec-fba5-4ec2-b848-495b192761e8-kube-api-access-7d4h4\") pod \"migrator-6865c9967-brq2q\" (UID: \"0e3799ec-fba5-4ec2-b848-495b192761e8\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-brq2q" Jul 24 13:43:26.943052 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:26.943021 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-brq2q" Jul 24 13:43:27.082987 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:27.082776 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-brq2q"] Jul 24 13:43:27.211392 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:27.211354 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:27.211569 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:27.211524 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 13:43:27.211634 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:27.211619 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls podName:3d96004d-7277-4a7e-9ec7-a87faa9526fd nodeName:}" failed. No retries permitted until 2026-07-24 13:43:28.211598395 +0000 UTC m=+41.898359790 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fvvgk" (UID: "3d96004d-7277-4a7e-9ec7-a87faa9526fd") : secret "insights-runtime-extractor-tls" not found Jul 24 13:43:27.211706 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:27.211689 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:27.211797 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:27.211784 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 13:43:27.211913 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:27.211896 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-rbac-proxy-cm podName:3d96004d-7277-4a7e-9ec7-a87faa9526fd nodeName:}" failed. No retries permitted until 2026-07-24 13:43:28.211878287 +0000 UTC m=+41.898639683 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fvvgk" (UID: "3d96004d-7277-4a7e-9ec7-a87faa9526fd") : configmap "kube-rbac-proxy" not found Jul 24 13:43:27.868077 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:27.868053 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5mnw5_5513d77f-a9df-4125-bd63-d8f9df7b7e93/dns-node-resolver/0.log" Jul 24 13:43:28.062687 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:28.062655 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-brq2q" event={"ID":"0e3799ec-fba5-4ec2-b848-495b192761e8","Type":"ContainerStarted","Data":"eb70ef860e9a474205ca1b76a973c208ebf5e03f206523c5299b5b1f0fd5fffe"} Jul 24 13:43:28.221880 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:28.221771 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:28.221880 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:28.221875 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:28.222281 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:28.221909 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 13:43:28.222281 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:28.221974 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-rbac-proxy-cm podName:3d96004d-7277-4a7e-9ec7-a87faa9526fd nodeName:}" failed. No retries permitted until 2026-07-24 13:43:30.22195933 +0000 UTC m=+43.908720725 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fvvgk" (UID: "3d96004d-7277-4a7e-9ec7-a87faa9526fd") : configmap "kube-rbac-proxy" not found Jul 24 13:43:28.222281 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:28.221986 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 13:43:28.222281 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:28.222042 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls podName:3d96004d-7277-4a7e-9ec7-a87faa9526fd nodeName:}" failed. No retries permitted until 2026-07-24 13:43:30.222026634 +0000 UTC m=+43.908788029 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fvvgk" (UID: "3d96004d-7277-4a7e-9ec7-a87faa9526fd") : secret "insights-runtime-extractor-tls" not found Jul 24 13:43:28.868351 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:28.868326 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j5cfd_9df22139-a68a-421b-807c-e6afb4bb691e/node-ca/0.log" Jul 24 13:43:29.066287 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.066254 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-brq2q" event={"ID":"0e3799ec-fba5-4ec2-b848-495b192761e8","Type":"ContainerStarted","Data":"5e2bd2c148be830d205282aae50e3650fc2e7804f617a1314e2d2f4600a1cdd3"} Jul 24 13:43:29.066427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.066306 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-brq2q" event={"ID":"0e3799ec-fba5-4ec2-b848-495b192761e8","Type":"ContainerStarted","Data":"cdb9ada94f091f3092d8eceb12090c80e38743453fb91606377e3bd437290e42"} Jul 24 13:43:29.079831 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.079757 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-brq2q" podStartSLOduration=1.621106203 podStartE2EDuration="3.079740781s" totalImagesPullingTime="1.458634578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:43:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:43:29.079398923 +0000 UTC m=+42.766160342" watchObservedRunningTime="2026-07-24 13:43:29.079740781 +0000 UTC m=+42.766502198" Jul 24 13:43:29.194743 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.194655 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-spxvh"] Jul 24 13:43:29.213111 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.213082 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-spxvh"] Jul 24 13:43:29.213261 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.213199 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-spxvh" Jul 24 13:43:29.215338 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.215315 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 24 13:43:29.215460 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.215375 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 24 13:43:29.215615 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.215595 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-gnm87\"" Jul 24 13:43:29.215685 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.215632 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 24 13:43:29.215734 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.215638 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 24 13:43:29.330380 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.330345 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c4034608-f3c8-40ab-b80e-fed8b859b2cd-signing-cabundle\") pod \"service-ca-658d7b7c78-spxvh\" (UID: \"c4034608-f3c8-40ab-b80e-fed8b859b2cd\") " pod="openshift-service-ca/service-ca-658d7b7c78-spxvh" Jul 24 13:43:29.330769 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.330393 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s8kq\" (UniqueName: \"kubernetes.io/projected/c4034608-f3c8-40ab-b80e-fed8b859b2cd-kube-api-access-5s8kq\") pod \"service-ca-658d7b7c78-spxvh\" (UID: \"c4034608-f3c8-40ab-b80e-fed8b859b2cd\") " pod="openshift-service-ca/service-ca-658d7b7c78-spxvh" Jul 24 13:43:29.330769 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.330481 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c4034608-f3c8-40ab-b80e-fed8b859b2cd-signing-key\") pod \"service-ca-658d7b7c78-spxvh\" (UID: \"c4034608-f3c8-40ab-b80e-fed8b859b2cd\") " pod="openshift-service-ca/service-ca-658d7b7c78-spxvh" Jul 24 13:43:29.431649 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.431610 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s8kq\" (UniqueName: \"kubernetes.io/projected/c4034608-f3c8-40ab-b80e-fed8b859b2cd-kube-api-access-5s8kq\") pod \"service-ca-658d7b7c78-spxvh\" (UID: \"c4034608-f3c8-40ab-b80e-fed8b859b2cd\") " pod="openshift-service-ca/service-ca-658d7b7c78-spxvh" Jul 24 13:43:29.431850 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.431680 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c4034608-f3c8-40ab-b80e-fed8b859b2cd-signing-key\") pod \"service-ca-658d7b7c78-spxvh\" (UID: \"c4034608-f3c8-40ab-b80e-fed8b859b2cd\") " pod="openshift-service-ca/service-ca-658d7b7c78-spxvh" Jul 24 13:43:29.431850 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.431742 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c4034608-f3c8-40ab-b80e-fed8b859b2cd-signing-cabundle\") pod \"service-ca-658d7b7c78-spxvh\" (UID: \"c4034608-f3c8-40ab-b80e-fed8b859b2cd\") " pod="openshift-service-ca/service-ca-658d7b7c78-spxvh" Jul 24 13:43:29.432383 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.432362 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c4034608-f3c8-40ab-b80e-fed8b859b2cd-signing-cabundle\") pod \"service-ca-658d7b7c78-spxvh\" (UID: \"c4034608-f3c8-40ab-b80e-fed8b859b2cd\") " pod="openshift-service-ca/service-ca-658d7b7c78-spxvh" Jul 24 13:43:29.434291 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.434268 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c4034608-f3c8-40ab-b80e-fed8b859b2cd-signing-key\") pod \"service-ca-658d7b7c78-spxvh\" (UID: \"c4034608-f3c8-40ab-b80e-fed8b859b2cd\") " pod="openshift-service-ca/service-ca-658d7b7c78-spxvh" Jul 24 13:43:29.444301 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.444267 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s8kq\" (UniqueName: \"kubernetes.io/projected/c4034608-f3c8-40ab-b80e-fed8b859b2cd-kube-api-access-5s8kq\") pod \"service-ca-658d7b7c78-spxvh\" (UID: \"c4034608-f3c8-40ab-b80e-fed8b859b2cd\") " pod="openshift-service-ca/service-ca-658d7b7c78-spxvh" Jul 24 13:43:29.521939 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.521843 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-spxvh" Jul 24 13:43:29.532285 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.532252 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:29.532382 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.532338 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:29.532440 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:29.532419 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:43:29.532470 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:29.532444 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:43:29.532470 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:29.532454 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-644fb596d7-kjvss: secret "image-registry-tls" not found Jul 24 13:43:29.532530 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:29.532490 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls podName:9e7565d1-c59f-4fd9-adcd-09042dd2345e nodeName:}" failed. No retries permitted until 2026-07-24 13:43:37.532471938 +0000 UTC m=+51.219233356 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls") pod "dns-default-h2vp7" (UID: "9e7565d1-c59f-4fd9-adcd-09042dd2345e") : secret "dns-default-metrics-tls" not found Jul 24 13:43:29.532530 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:29.532505 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls podName:fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73 nodeName:}" failed. No retries permitted until 2026-07-24 13:43:37.532498629 +0000 UTC m=+51.219260025 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls") pod "image-registry-644fb596d7-kjvss" (UID: "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73") : secret "image-registry-tls" not found Jul 24 13:43:29.664973 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.664941 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-spxvh"] Jul 24 13:43:29.668379 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:29.668348 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-25lb7_849e0755-ade6-4136-9394-017410ace217/serve-healthcheck-canary/0.log" Jul 24 13:43:29.669188 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:43:29.669166 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4034608_f3c8_40ab_b80e_fed8b859b2cd.slice/crio-bcb8c46e5bb889ee60747187d1d40366784d0d9826fbb22d5490ab226eb93d7e WatchSource:0}: Error finding container bcb8c46e5bb889ee60747187d1d40366784d0d9826fbb22d5490ab226eb93d7e: Status 404 returned error can't find the container with id bcb8c46e5bb889ee60747187d1d40366784d0d9826fbb22d5490ab226eb93d7e Jul 24 13:43:30.069987 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:30.069950 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-spxvh" event={"ID":"c4034608-f3c8-40ab-b80e-fed8b859b2cd","Type":"ContainerStarted","Data":"bcb8c46e5bb889ee60747187d1d40366784d0d9826fbb22d5490ab226eb93d7e"} Jul 24 13:43:30.238175 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:30.238140 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:30.238371 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:30.238313 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 13:43:30.238371 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:30.238354 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:30.238492 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:30.238381 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls podName:3d96004d-7277-4a7e-9ec7-a87faa9526fd nodeName:}" failed. No retries permitted until 2026-07-24 13:43:34.238361942 +0000 UTC m=+47.925123338 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fvvgk" (UID: "3d96004d-7277-4a7e-9ec7-a87faa9526fd") : secret "insights-runtime-extractor-tls" not found Jul 24 13:43:30.238492 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:30.238408 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 13:43:30.238492 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:30.238443 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-rbac-proxy-cm podName:3d96004d-7277-4a7e-9ec7-a87faa9526fd nodeName:}" failed. No retries permitted until 2026-07-24 13:43:34.238433628 +0000 UTC m=+47.925195025 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fvvgk" (UID: "3d96004d-7277-4a7e-9ec7-a87faa9526fd") : configmap "kube-rbac-proxy" not found Jul 24 13:43:32.076122 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:32.076087 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-spxvh" event={"ID":"c4034608-f3c8-40ab-b80e-fed8b859b2cd","Type":"ContainerStarted","Data":"4306a753b007e427bec34089d0d6cd28732a69898ad8066885428405796b0816"} Jul 24 13:43:32.090782 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:32.090725 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-spxvh" podStartSLOduration=0.878750802 podStartE2EDuration="3.090710308s" totalImagesPullingTime="2.211959506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:43:32.089646952 +0000 UTC m=+45.776408369" watchObservedRunningTime="2026-07-24 13:43:32.090710308 +0000 UTC m=+45.777471724" Jul 24 13:43:34.280274 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:34.280235 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:34.280669 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:34.280331 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:34.280669 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:34.280408 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 13:43:34.280669 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:43:34.280478 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls podName:3d96004d-7277-4a7e-9ec7-a87faa9526fd nodeName:}" failed. No retries permitted until 2026-07-24 13:43:42.280459257 +0000 UTC m=+55.967220658 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fvvgk" (UID: "3d96004d-7277-4a7e-9ec7-a87faa9526fd") : secret "insights-runtime-extractor-tls" not found Jul 24 13:43:34.280802 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:34.280785 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d96004d-7277-4a7e-9ec7-a87faa9526fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:37.605687 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:37.605647 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:37.606114 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:37.605716 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:37.608136 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:37.608114 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e7565d1-c59f-4fd9-adcd-09042dd2345e-metrics-tls\") pod \"dns-default-h2vp7\" (UID: \"9e7565d1-c59f-4fd9-adcd-09042dd2345e\") " pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:37.608256 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:37.608172 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls\") pod \"image-registry-644fb596d7-kjvss\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:37.628125 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:37.628100 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:37.639261 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:37.639236 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:37.803070 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:37.803037 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-644fb596d7-kjvss"] Jul 24 13:43:37.806208 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:43:37.806181 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdbc1a29_cdfb_4789_abf6_f6bfbe22ce73.slice/crio-311f92ae8563ca903a3f4edcbbf564238aafc0799dca2ad42e2cd109b19ad470 WatchSource:0}: Error finding container 311f92ae8563ca903a3f4edcbbf564238aafc0799dca2ad42e2cd109b19ad470: Status 404 returned error can't find the container with id 311f92ae8563ca903a3f4edcbbf564238aafc0799dca2ad42e2cd109b19ad470 Jul 24 13:43:37.818074 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:37.818050 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h2vp7"] Jul 24 13:43:37.821618 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:43:37.821595 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e7565d1_c59f_4fd9_adcd_09042dd2345e.slice/crio-4988bb25caac47b9a29fd0e22447c584641624b37bb74da14d077a43e8ece5a4 WatchSource:0}: Error finding container 4988bb25caac47b9a29fd0e22447c584641624b37bb74da14d077a43e8ece5a4: Status 404 returned error can't find the container with id 4988bb25caac47b9a29fd0e22447c584641624b37bb74da14d077a43e8ece5a4 Jul 24 13:43:38.094346 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:38.094308 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h2vp7" event={"ID":"9e7565d1-c59f-4fd9-adcd-09042dd2345e","Type":"ContainerStarted","Data":"4988bb25caac47b9a29fd0e22447c584641624b37bb74da14d077a43e8ece5a4"} Jul 24 13:43:38.095945 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:38.095916 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644fb596d7-kjvss" event={"ID":"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73","Type":"ContainerStarted","Data":"fdd64ae9034f9ca86f036d7ef78201ac14dc5fa2f65488b72e50944bd92964e4"} Jul 24 13:43:38.096058 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:38.095965 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644fb596d7-kjvss" event={"ID":"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73","Type":"ContainerStarted","Data":"311f92ae8563ca903a3f4edcbbf564238aafc0799dca2ad42e2cd109b19ad470"} Jul 24 13:43:38.096115 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:38.096062 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:43:38.115439 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:38.115357 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-644fb596d7-kjvss" podStartSLOduration=29.11534291 podStartE2EDuration="29.11534291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:43:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:43:38.114471084 +0000 UTC m=+51.801232504" watchObservedRunningTime="2026-07-24 13:43:38.11534291 +0000 UTC m=+51.802104326" Jul 24 13:43:41.106500 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:41.106466 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h2vp7" event={"ID":"9e7565d1-c59f-4fd9-adcd-09042dd2345e","Type":"ContainerStarted","Data":"c92c3a45813e0154277fbf1826633c6e49a40c8786ce9e9b4b3115e18206dd42"} Jul 24 13:43:41.106911 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:41.106519 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h2vp7" event={"ID":"9e7565d1-c59f-4fd9-adcd-09042dd2345e","Type":"ContainerStarted","Data":"48abb6cfbc7824e415761fe43c3e5d7479833c44a8d70f14819453b9d30fa7e0"} Jul 24 13:43:41.106911 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:41.106616 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:41.125850 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:41.125777 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-h2vp7" podStartSLOduration=17.860701033 podStartE2EDuration="20.125763462s" totalImagesPullingTime="2.265062429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:43:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:43:41.124836051 +0000 UTC m=+54.811597465" watchObservedRunningTime="2026-07-24 13:43:41.125763462 +0000 UTC m=+54.812524878" Jul 24 13:43:42.347241 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:42.347205 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:42.349727 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:42.349702 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d96004d-7277-4a7e-9ec7-a87faa9526fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvvgk\" (UID: \"3d96004d-7277-4a7e-9ec7-a87faa9526fd\") " pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:42.408435 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:42.408399 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fvvgk" Jul 24 13:43:42.535450 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:42.535411 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fvvgk"] Jul 24 13:43:42.538794 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:43:42.538768 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d96004d_7277_4a7e_9ec7_a87faa9526fd.slice/crio-93df56a29c5b78610d71cae1fca53023154e8b62b591d31b1f03502d9fe0659f WatchSource:0}: Error finding container 93df56a29c5b78610d71cae1fca53023154e8b62b591d31b1f03502d9fe0659f: Status 404 returned error can't find the container with id 93df56a29c5b78610d71cae1fca53023154e8b62b591d31b1f03502d9fe0659f Jul 24 13:43:43.114037 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:43.114000 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fvvgk" event={"ID":"3d96004d-7277-4a7e-9ec7-a87faa9526fd","Type":"ContainerStarted","Data":"93df56a29c5b78610d71cae1fca53023154e8b62b591d31b1f03502d9fe0659f"} Jul 24 13:43:47.035170 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:47.035138 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h8gk4" Jul 24 13:43:49.139544 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:49.139493 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fvvgk" event={"ID":"3d96004d-7277-4a7e-9ec7-a87faa9526fd","Type":"ContainerStarted","Data":"a88b46dbc27696a7c3f5a630330731ed6c53680e16104917cf0a645ffea3fb53"} Jul 24 13:43:51.064706 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.064659 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n"] Jul 24 13:43:51.071533 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.071493 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-hhx58"] Jul 24 13:43:51.072882 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.071842 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n" Jul 24 13:43:51.074623 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.074590 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-chx9k\"" Jul 24 13:43:51.075662 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.074864 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 24 13:43:51.075662 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.075055 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 24 13:43:51.075662 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.075272 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 24 13:43:51.075662 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.075514 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 24 13:43:51.076092 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.075860 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2"] Jul 24 13:43:51.076092 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.075991 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-hhx58" Jul 24 13:43:51.077999 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.077981 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 24 13:43:51.078411 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.078387 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jhk8h\"" Jul 24 13:43:51.078532 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.078500 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 24 13:43:51.080242 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.080222 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-644fb596d7-kjvss"] Jul 24 13:43:51.080839 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.080462 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.081873 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.081850 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-hhx58"] Jul 24 13:43:51.083934 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.083913 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 24 13:43:51.087989 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.087583 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 24 13:43:51.087989 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.087828 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 24 13:43:51.087989 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.087900 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 24 13:43:51.092496 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.091236 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n"] Jul 24 13:43:51.092496 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.092446 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2"] Jul 24 13:43:51.112499 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.111602 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-h2vp7" Jul 24 13:43:51.123742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.123711 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd70d372-a17a-4ba2-8827-34a1adbdbc36-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.123933 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.123752 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7475\" (UniqueName: \"kubernetes.io/projected/bd70d372-a17a-4ba2-8827-34a1adbdbc36-kube-api-access-m7475\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.123933 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.123784 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/baf4fefa-586a-4887-b3ce-9fd95f191b83-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-hhx58\" (UID: \"baf4fefa-586a-4887-b3ce-9fd95f191b83\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-hhx58" Jul 24 13:43:51.123933 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.123863 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnnt2\" (UniqueName: \"kubernetes.io/projected/8d136751-62a9-49fc-82d6-1803865583c3-kube-api-access-jnnt2\") pod \"managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n\" (UID: \"8d136751-62a9-49fc-82d6-1803865583c3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n" Jul 24 13:43:51.124117 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.123935 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/baf4fefa-586a-4887-b3ce-9fd95f191b83-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-hhx58\" (UID: \"baf4fefa-586a-4887-b3ce-9fd95f191b83\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-hhx58" Jul 24 13:43:51.124117 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.123987 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bd70d372-a17a-4ba2-8827-34a1adbdbc36-ca\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.124117 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.124034 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d136751-62a9-49fc-82d6-1803865583c3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n\" (UID: \"8d136751-62a9-49fc-82d6-1803865583c3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n" Jul 24 13:43:51.124117 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.124064 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd70d372-a17a-4ba2-8827-34a1adbdbc36-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.124117 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.124091 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bd70d372-a17a-4ba2-8827-34a1adbdbc36-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.124405 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.124125 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bd70d372-a17a-4ba2-8827-34a1adbdbc36-hub\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.165720 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.165682 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7fb94d7fbb-6kwxh"] Jul 24 13:43:51.172182 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.172155 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.185856 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.185794 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fb94d7fbb-6kwxh"] Jul 24 13:43:51.225254 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.225221 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65600164-6849-4bb2-83c2-1be903b7b59a-trusted-ca\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.225472 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.225263 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65600164-6849-4bb2-83c2-1be903b7b59a-registry-certificates\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.225472 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.225330 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65600164-6849-4bb2-83c2-1be903b7b59a-installation-pull-secrets\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.225472 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.225360 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnnt2\" (UniqueName: \"kubernetes.io/projected/8d136751-62a9-49fc-82d6-1803865583c3-kube-api-access-jnnt2\") pod \"managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n\" (UID: \"8d136751-62a9-49fc-82d6-1803865583c3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n" Jul 24 13:43:51.225472 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.225413 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/baf4fefa-586a-4887-b3ce-9fd95f191b83-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-hhx58\" (UID: \"baf4fefa-586a-4887-b3ce-9fd95f191b83\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-hhx58" Jul 24 13:43:51.225679 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.225473 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bd70d372-a17a-4ba2-8827-34a1adbdbc36-ca\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.225679 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.225508 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8862v\" (UniqueName: \"kubernetes.io/projected/65600164-6849-4bb2-83c2-1be903b7b59a-kube-api-access-8862v\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.225679 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.225544 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d136751-62a9-49fc-82d6-1803865583c3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n\" (UID: \"8d136751-62a9-49fc-82d6-1803865583c3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n" Jul 24 13:43:51.225679 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.225572 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd70d372-a17a-4ba2-8827-34a1adbdbc36-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.225679 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.225638 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bd70d372-a17a-4ba2-8827-34a1adbdbc36-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.225679 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.225670 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65600164-6849-4bb2-83c2-1be903b7b59a-registry-tls\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.226014 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.225738 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bd70d372-a17a-4ba2-8827-34a1adbdbc36-hub\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.226014 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.225783 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65600164-6849-4bb2-83c2-1be903b7b59a-image-registry-private-configuration\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.226014 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.225835 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65600164-6849-4bb2-83c2-1be903b7b59a-ca-trust-extracted\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.226014 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.225921 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd70d372-a17a-4ba2-8827-34a1adbdbc36-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.226014 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.225966 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7475\" (UniqueName: \"kubernetes.io/projected/bd70d372-a17a-4ba2-8827-34a1adbdbc36-kube-api-access-m7475\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.226014 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.226005 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/baf4fefa-586a-4887-b3ce-9fd95f191b83-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-hhx58\" (UID: \"baf4fefa-586a-4887-b3ce-9fd95f191b83\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-hhx58" Jul 24 13:43:51.226461 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.226062 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65600164-6849-4bb2-83c2-1be903b7b59a-bound-sa-token\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.226523 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.226500 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd70d372-a17a-4ba2-8827-34a1adbdbc36-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.227246 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.227217 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/baf4fefa-586a-4887-b3ce-9fd95f191b83-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-hhx58\" (UID: \"baf4fefa-586a-4887-b3ce-9fd95f191b83\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-hhx58" Jul 24 13:43:51.228688 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.228644 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bd70d372-a17a-4ba2-8827-34a1adbdbc36-ca\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.229911 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.229379 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd70d372-a17a-4ba2-8827-34a1adbdbc36-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.229911 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.229498 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/baf4fefa-586a-4887-b3ce-9fd95f191b83-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-hhx58\" (UID: \"baf4fefa-586a-4887-b3ce-9fd95f191b83\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-hhx58" Jul 24 13:43:51.229911 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.229849 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bd70d372-a17a-4ba2-8827-34a1adbdbc36-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.229911 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.229872 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bd70d372-a17a-4ba2-8827-34a1adbdbc36-hub\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.231895 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.231876 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d136751-62a9-49fc-82d6-1803865583c3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n\" (UID: \"8d136751-62a9-49fc-82d6-1803865583c3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n" Jul 24 13:43:51.248043 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.248005 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7475\" (UniqueName: \"kubernetes.io/projected/bd70d372-a17a-4ba2-8827-34a1adbdbc36-kube-api-access-m7475\") pod \"cluster-proxy-proxy-agent-5954f85695-cf2d2\" (UID: \"bd70d372-a17a-4ba2-8827-34a1adbdbc36\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.248190 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.248088 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnnt2\" (UniqueName: \"kubernetes.io/projected/8d136751-62a9-49fc-82d6-1803865583c3-kube-api-access-jnnt2\") pod \"managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n\" (UID: \"8d136751-62a9-49fc-82d6-1803865583c3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n" Jul 24 13:43:51.326483 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.326402 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65600164-6849-4bb2-83c2-1be903b7b59a-image-registry-private-configuration\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.326483 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.326436 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65600164-6849-4bb2-83c2-1be903b7b59a-ca-trust-extracted\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.326700 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.326492 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65600164-6849-4bb2-83c2-1be903b7b59a-bound-sa-token\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.326700 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.326516 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65600164-6849-4bb2-83c2-1be903b7b59a-trusted-ca\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.326700 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.326548 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65600164-6849-4bb2-83c2-1be903b7b59a-registry-certificates\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.326700 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.326568 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65600164-6849-4bb2-83c2-1be903b7b59a-installation-pull-secrets\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.326700 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.326618 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8862v\" (UniqueName: \"kubernetes.io/projected/65600164-6849-4bb2-83c2-1be903b7b59a-kube-api-access-8862v\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.326700 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.326661 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65600164-6849-4bb2-83c2-1be903b7b59a-registry-tls\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.327276 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.327230 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65600164-6849-4bb2-83c2-1be903b7b59a-ca-trust-extracted\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.327836 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.327792 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65600164-6849-4bb2-83c2-1be903b7b59a-trusted-ca\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.328384 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.328351 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65600164-6849-4bb2-83c2-1be903b7b59a-registry-certificates\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.329584 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.329523 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65600164-6849-4bb2-83c2-1be903b7b59a-image-registry-private-configuration\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.329695 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.329675 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65600164-6849-4bb2-83c2-1be903b7b59a-installation-pull-secrets\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.330037 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.330016 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65600164-6849-4bb2-83c2-1be903b7b59a-registry-tls\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.339021 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.338967 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65600164-6849-4bb2-83c2-1be903b7b59a-bound-sa-token\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.339310 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.339275 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8862v\" (UniqueName: \"kubernetes.io/projected/65600164-6849-4bb2-83c2-1be903b7b59a-kube-api-access-8862v\") pod \"image-registry-7fb94d7fbb-6kwxh\" (UID: \"65600164-6849-4bb2-83c2-1be903b7b59a\") " pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.401891 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.401859 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n" Jul 24 13:43:51.411831 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.411785 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-hhx58" Jul 24 13:43:51.418891 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.418867 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" Jul 24 13:43:51.498631 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.498595 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:51.802077 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:51.802045 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n"] Jul 24 13:43:51.804597 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:43:51.804570 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d136751_62a9_49fc_82d6_1803865583c3.slice/crio-3c0921b5b6465c8c029ef7e5e38ae126b1880a4d30dd8d6611736aaa45f3a8e3 WatchSource:0}: Error finding container 3c0921b5b6465c8c029ef7e5e38ae126b1880a4d30dd8d6611736aaa45f3a8e3: Status 404 returned error can't find the container with id 3c0921b5b6465c8c029ef7e5e38ae126b1880a4d30dd8d6611736aaa45f3a8e3 Jul 24 13:43:52.020978 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.020950 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-hhx58"] Jul 24 13:43:52.023566 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:43:52.023530 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaf4fefa_586a_4887_b3ce_9fd95f191b83.slice/crio-b280aeb3d9b20e23a8e979c570f18788bd3377907950b0dd6eb767fb23c7f572 WatchSource:0}: Error finding container b280aeb3d9b20e23a8e979c570f18788bd3377907950b0dd6eb767fb23c7f572: Status 404 returned error can't find the container with id b280aeb3d9b20e23a8e979c570f18788bd3377907950b0dd6eb767fb23c7f572 Jul 24 13:43:52.024943 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.024880 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2"] Jul 24 13:43:52.026158 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.026126 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fb94d7fbb-6kwxh"] Jul 24 13:43:52.028586 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:43:52.028558 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65600164_6849_4bb2_83c2_1be903b7b59a.slice/crio-6fee04d4f31a30088c9a2cbdca2d281b65a299d86f6ed35426833472ee3c23ba WatchSource:0}: Error finding container 6fee04d4f31a30088c9a2cbdca2d281b65a299d86f6ed35426833472ee3c23ba: Status 404 returned error can't find the container with id 6fee04d4f31a30088c9a2cbdca2d281b65a299d86f6ed35426833472ee3c23ba Jul 24 13:43:52.030453 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:43:52.030430 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd70d372_a17a_4ba2_8827_34a1adbdbc36.slice/crio-3be133aedb42bc3678026661888ae1298948ab0e1b8212aa0b998ec43dee6d7b WatchSource:0}: Error finding container 3be133aedb42bc3678026661888ae1298948ab0e1b8212aa0b998ec43dee6d7b: Status 404 returned error can't find the container with id 3be133aedb42bc3678026661888ae1298948ab0e1b8212aa0b998ec43dee6d7b Jul 24 13:43:52.151253 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.151184 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" event={"ID":"bd70d372-a17a-4ba2-8827-34a1adbdbc36","Type":"ContainerStarted","Data":"3be133aedb42bc3678026661888ae1298948ab0e1b8212aa0b998ec43dee6d7b"} Jul 24 13:43:52.152534 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.152513 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-hhx58" event={"ID":"baf4fefa-586a-4887-b3ce-9fd95f191b83","Type":"ContainerStarted","Data":"b280aeb3d9b20e23a8e979c570f18788bd3377907950b0dd6eb767fb23c7f572"} Jul 24 13:43:52.153985 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.153965 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" event={"ID":"65600164-6849-4bb2-83c2-1be903b7b59a","Type":"ContainerStarted","Data":"e446d6cdba2216b4d1e5fe80d2be04f086cfc6984af3230f08cf85750ca205e8"} Jul 24 13:43:52.154084 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.153999 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" event={"ID":"65600164-6849-4bb2-83c2-1be903b7b59a","Type":"ContainerStarted","Data":"6fee04d4f31a30088c9a2cbdca2d281b65a299d86f6ed35426833472ee3c23ba"} Jul 24 13:43:52.154207 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.154186 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:43:52.155767 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.155742 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fvvgk" event={"ID":"3d96004d-7277-4a7e-9ec7-a87faa9526fd","Type":"ContainerStarted","Data":"33cdbc6c8e8133668b512503a97629474131e6771ceb5f3a61774d9e7aae4f66"} Jul 24 13:43:52.156874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.156852 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n" event={"ID":"8d136751-62a9-49fc-82d6-1803865583c3","Type":"ContainerStarted","Data":"3c0921b5b6465c8c029ef7e5e38ae126b1880a4d30dd8d6611736aaa45f3a8e3"} Jul 24 13:43:52.172630 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.172581 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" podStartSLOduration=1.17256538 podStartE2EDuration="1.17256538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:43:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:43:52.171468349 +0000 UTC m=+65.858229765" watchObservedRunningTime="2026-07-24 13:43:52.17256538 +0000 UTC m=+65.859326791" Jul 24 13:43:52.537135 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.537056 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs\") pod \"network-metrics-daemon-vjcfb\" (UID: \"46030f6f-52fc-40f7-b5d4-7febe8bd998a\") " pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:52.539434 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.539406 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 13:43:52.551217 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.551181 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46030f6f-52fc-40f7-b5d4-7febe8bd998a-metrics-certs\") pod \"network-metrics-daemon-vjcfb\" (UID: \"46030f6f-52fc-40f7-b5d4-7febe8bd998a\") " pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:52.586092 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.585865 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cbzbq\"" Jul 24 13:43:52.595054 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.594791 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vjcfb" Jul 24 13:43:52.638733 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.638123 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp2nb\" (UniqueName: \"kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb\") pod \"network-check-target-9sfhz\" (UID: \"f311e199-3ddf-4650-8f2b-cde419d63d1a\") " pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:52.638733 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.638181 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret\") pod \"global-pull-secret-syncer-q7hvp\" (UID: \"533b42cc-ae61-4729-b5a1-e904f90f7fcc\") " pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:52.640700 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.640525 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 13:43:52.640700 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.640583 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 13:43:52.651654 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.651439 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 13:43:52.653469 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.653415 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/533b42cc-ae61-4729-b5a1-e904f90f7fcc-original-pull-secret\") pod \"global-pull-secret-syncer-q7hvp\" (UID: \"533b42cc-ae61-4729-b5a1-e904f90f7fcc\") " pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:52.665051 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.664048 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp2nb\" (UniqueName: \"kubernetes.io/projected/f311e199-3ddf-4650-8f2b-cde419d63d1a-kube-api-access-kp2nb\") pod \"network-check-target-9sfhz\" (UID: \"f311e199-3ddf-4650-8f2b-cde419d63d1a\") " pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:52.802775 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.802606 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vjcfb"] Jul 24 13:43:52.892967 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.892935 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l5jkr\"" Jul 24 13:43:52.897713 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.897687 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7hvp" Jul 24 13:43:52.901546 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:52.901526 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:43:53.130310 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:53.130220 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q7hvp"] Jul 24 13:43:53.156941 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:53.156668 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9sfhz"] Jul 24 13:43:53.164302 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:53.164229 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vjcfb" event={"ID":"46030f6f-52fc-40f7-b5d4-7febe8bd998a","Type":"ContainerStarted","Data":"24e4ec25d0935af4301827b3cb71980b72a769300a148573df06fd4a767ecce3"} Jul 24 13:43:53.870545 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:43:53.870401 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf311e199_3ddf_4650_8f2b_cde419d63d1a.slice/crio-9dc51f00c5deb1840b86e08ea7ffc5a099a4286087b0ee48b03e936674e6d647 WatchSource:0}: Error finding container 9dc51f00c5deb1840b86e08ea7ffc5a099a4286087b0ee48b03e936674e6d647: Status 404 returned error can't find the container with id 9dc51f00c5deb1840b86e08ea7ffc5a099a4286087b0ee48b03e936674e6d647 Jul 24 13:43:54.175382 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:54.175224 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-hhx58" event={"ID":"baf4fefa-586a-4887-b3ce-9fd95f191b83","Type":"ContainerStarted","Data":"32efbb97b2f5e2f97d74e922f4d9fcfa5433127e65c6fa838ebcf77bf8bb9156"} Jul 24 13:43:54.178299 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:54.178195 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q7hvp" event={"ID":"533b42cc-ae61-4729-b5a1-e904f90f7fcc","Type":"ContainerStarted","Data":"f7440c6d47166c73271a556c6eb0cb2c9034da3faa2e8c9a7f1d28ddbf7270bc"} Jul 24 13:43:54.180342 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:54.180291 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9sfhz" event={"ID":"f311e199-3ddf-4650-8f2b-cde419d63d1a","Type":"ContainerStarted","Data":"9dc51f00c5deb1840b86e08ea7ffc5a099a4286087b0ee48b03e936674e6d647"} Jul 24 13:43:54.191964 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:43:54.190989 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-hhx58" podStartSLOduration=1.28289003 podStartE2EDuration="3.190965647s" totalImagesPullingTime="1.908075617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:43:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:43:54.190001611 +0000 UTC m=+67.876763063" watchObservedRunningTime="2026-07-24 13:43:54.190965647 +0000 UTC m=+67.877727068" Jul 24 13:44:01.088936 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:01.088901 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:44:02.215460 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:02.215384 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q7hvp" event={"ID":"533b42cc-ae61-4729-b5a1-e904f90f7fcc","Type":"ContainerStarted","Data":"b30c80c9d36b2a3ad16c4472ff15638d7056cdd6390710b794abd78e2c7f3319"} Jul 24 13:44:02.217455 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:02.217420 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9sfhz" event={"ID":"f311e199-3ddf-4650-8f2b-cde419d63d1a","Type":"ContainerStarted","Data":"14887a82cdff0d3308191ead56233b9a5eed47ef2ad143c5babeb4b5b9e6ac4b"} Jul 24 13:44:02.217592 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:02.217578 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:44:02.219571 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:02.219549 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vjcfb" event={"ID":"46030f6f-52fc-40f7-b5d4-7febe8bd998a","Type":"ContainerStarted","Data":"ee002a5eb2fbec372477960b980dffdbb1bdd6cfdfa5ea303e1b9480a98bf785"} Jul 24 13:44:02.219702 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:02.219691 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vjcfb" event={"ID":"46030f6f-52fc-40f7-b5d4-7febe8bd998a","Type":"ContainerStarted","Data":"bdc1bd006f31697c874c3e9670ce6193b493dfb1606b8fa774d97a878ce9058e"} Jul 24 13:44:02.221856 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:02.221832 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" event={"ID":"bd70d372-a17a-4ba2-8827-34a1adbdbc36","Type":"ContainerStarted","Data":"84a44434c19c560a45d959242da3d92a0d86548e6451ca7e3fc6d4cec3d62080"} Jul 24 13:44:02.221972 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:02.221871 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" event={"ID":"bd70d372-a17a-4ba2-8827-34a1adbdbc36","Type":"ContainerStarted","Data":"d535b30575e585e14e4b0b4af90e1b597897e3e4babd5e8bdec727e7690d417c"} Jul 24 13:44:02.226210 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:02.225632 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fvvgk" event={"ID":"3d96004d-7277-4a7e-9ec7-a87faa9526fd","Type":"ContainerStarted","Data":"fa38f59394da8c9908fa53e1210ee9b13adfc12bd1bb9c1c480fc1cdd5bc5094"} Jul 24 13:44:02.227801 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:02.227779 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n" event={"ID":"8d136751-62a9-49fc-82d6-1803865583c3","Type":"ContainerStarted","Data":"a849a40fc617d27599c1989215dd24b94bb8432ec7217281a37a7b2bf8a194ea"} Jul 24 13:44:02.230626 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:02.230573 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-q7hvp" podStartSLOduration=68.127623646 podStartE2EDuration="1m16.230555631s" totalImagesPullingTime="8.102931985s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:44:02.230239383 +0000 UTC m=+75.917000791" watchObservedRunningTime="2026-07-24 13:44:02.230555631 +0000 UTC m=+75.917317049" Jul 24 13:44:02.245828 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:02.245744 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9sfhz" podStartSLOduration=68.153471326 podStartE2EDuration="1m16.245726666s" totalImagesPullingTime="8.09225534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:44:02.243422718 +0000 UTC m=+75.930184147" watchObservedRunningTime="2026-07-24 13:44:02.245726666 +0000 UTC m=+75.932488089" Jul 24 13:44:02.260850 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:02.260771 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fvvgk" podStartSLOduration=17.09896874 podStartE2EDuration="36.260750845s" totalImagesPullingTime="19.161782105s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:43:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:44:02.259485913 +0000 UTC m=+75.946247333" watchObservedRunningTime="2026-07-24 13:44:02.260750845 +0000 UTC m=+75.947512263" Jul 24 13:44:02.274373 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:02.274307 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5797cbf9d6-lmt2n" podStartSLOduration=1.170843246 podStartE2EDuration="11.274286365s" totalImagesPullingTime="10.103443119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:43:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:44:02.272790768 +0000 UTC m=+75.959552186" watchObservedRunningTime="2026-07-24 13:44:02.274286365 +0000 UTC m=+75.961047783" Jul 24 13:44:02.287542 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:02.287483 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vjcfb" podStartSLOduration=67.193738972 podStartE2EDuration="1m16.2874614s" totalImagesPullingTime="9.093722428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:44:02.287187677 +0000 UTC m=+75.973949093" watchObservedRunningTime="2026-07-24 13:44:02.2874614 +0000 UTC m=+75.974222819" Jul 24 13:44:03.234003 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:03.233958 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" event={"ID":"bd70d372-a17a-4ba2-8827-34a1adbdbc36","Type":"ContainerStarted","Data":"70cf487a1a969545f7cf441acac83e7c32df2902ab0c56d5ddfcb0b485143fd6"} Jul 24 13:44:03.270595 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:03.270280 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5954f85695-cf2d2" podStartSLOduration=2.348487323 podStartE2EDuration="12.270260352s" totalImagesPullingTime="9.921773029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:43:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:44:03.269277099 +0000 UTC m=+76.956038529" watchObservedRunningTime="2026-07-24 13:44:03.270260352 +0000 UTC m=+76.957021760" Jul 24 13:44:12.104712 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.104571 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-npd7p"] Jul 24 13:44:12.109283 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.109252 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.112082 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.112049 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 24 13:44:12.112082 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.112074 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 24 13:44:12.112278 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.112055 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 24 13:44:12.112502 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.112484 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 24 13:44:12.112602 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.112523 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 24 13:44:12.112602 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.112485 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q8wnd\"" Jul 24 13:44:12.112602 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.112485 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 24 13:44:12.199333 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.199287 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a88ae537-f807-4aa1-ab8e-da01013105ad-node-exporter-accelerators-collector-config\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.199527 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.199346 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t88v\" (UniqueName: \"kubernetes.io/projected/a88ae537-f807-4aa1-ab8e-da01013105ad-kube-api-access-4t88v\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.199527 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.199414 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a88ae537-f807-4aa1-ab8e-da01013105ad-root\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.199527 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.199454 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a88ae537-f807-4aa1-ab8e-da01013105ad-metrics-client-ca\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.199527 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.199481 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a88ae537-f807-4aa1-ab8e-da01013105ad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.199527 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.199504 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a88ae537-f807-4aa1-ab8e-da01013105ad-sys\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.199777 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.199528 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a88ae537-f807-4aa1-ab8e-da01013105ad-node-exporter-textfile\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.199777 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.199603 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a88ae537-f807-4aa1-ab8e-da01013105ad-node-exporter-wtmp\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.199777 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.199665 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a88ae537-f807-4aa1-ab8e-da01013105ad-node-exporter-tls\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.300822 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.300767 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a88ae537-f807-4aa1-ab8e-da01013105ad-root\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.301019 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.300837 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a88ae537-f807-4aa1-ab8e-da01013105ad-metrics-client-ca\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.301019 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.300865 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a88ae537-f807-4aa1-ab8e-da01013105ad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.301019 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.300891 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a88ae537-f807-4aa1-ab8e-da01013105ad-sys\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.301019 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.300892 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a88ae537-f807-4aa1-ab8e-da01013105ad-root\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.301019 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.300919 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a88ae537-f807-4aa1-ab8e-da01013105ad-node-exporter-textfile\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.301019 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.300947 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a88ae537-f807-4aa1-ab8e-da01013105ad-node-exporter-wtmp\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.301019 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.300958 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a88ae537-f807-4aa1-ab8e-da01013105ad-sys\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.301019 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.301006 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a88ae537-f807-4aa1-ab8e-da01013105ad-node-exporter-tls\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.301427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.301042 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a88ae537-f807-4aa1-ab8e-da01013105ad-node-exporter-accelerators-collector-config\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.301427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.301075 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t88v\" (UniqueName: \"kubernetes.io/projected/a88ae537-f807-4aa1-ab8e-da01013105ad-kube-api-access-4t88v\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.301427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.301088 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a88ae537-f807-4aa1-ab8e-da01013105ad-node-exporter-wtmp\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.301590 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.301431 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a88ae537-f807-4aa1-ab8e-da01013105ad-node-exporter-textfile\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.301590 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.301542 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a88ae537-f807-4aa1-ab8e-da01013105ad-metrics-client-ca\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.301847 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.301803 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a88ae537-f807-4aa1-ab8e-da01013105ad-node-exporter-accelerators-collector-config\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.304089 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.304056 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a88ae537-f807-4aa1-ab8e-da01013105ad-node-exporter-tls\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.304194 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.304158 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a88ae537-f807-4aa1-ab8e-da01013105ad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.309822 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.309789 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t88v\" (UniqueName: \"kubernetes.io/projected/a88ae537-f807-4aa1-ab8e-da01013105ad-kube-api-access-4t88v\") pod \"node-exporter-npd7p\" (UID: \"a88ae537-f807-4aa1-ab8e-da01013105ad\") " pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:12.420670 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:12.420541 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-npd7p" Jul 24 13:44:13.170960 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:13.170931 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7fb94d7fbb-6kwxh" Jul 24 13:44:13.271986 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:13.271946 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-npd7p" event={"ID":"a88ae537-f807-4aa1-ab8e-da01013105ad","Type":"ContainerStarted","Data":"c7e3c5bcdc70f51e374e1e919b8fa66b8decfd9faa2ea69596134cc2d057e13a"} Jul 24 13:44:14.277004 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:14.276972 2557 generic.go:356] "Generic (PLEG): container finished" podID="a88ae537-f807-4aa1-ab8e-da01013105ad" containerID="26aacdd8537fb193b70b299603fda8c67d79ff5d19c313504d713c420a28f5a1" exitCode=0 Jul 24 13:44:14.277391 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:14.277039 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-npd7p" event={"ID":"a88ae537-f807-4aa1-ab8e-da01013105ad","Type":"ContainerDied","Data":"26aacdd8537fb193b70b299603fda8c67d79ff5d19c313504d713c420a28f5a1"} Jul 24 13:44:15.282444 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:15.282404 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-npd7p" event={"ID":"a88ae537-f807-4aa1-ab8e-da01013105ad","Type":"ContainerStarted","Data":"b44ffb819d0f3c9be1f3bcb017c7601a48d50f0315f312d0acad6bd5ad346e39"} Jul 24 13:44:15.282444 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:15.282456 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-npd7p" event={"ID":"a88ae537-f807-4aa1-ab8e-da01013105ad","Type":"ContainerStarted","Data":"4541a585d0395213af23301f1c389d565715870fa51a7309238b428a62835509"} Jul 24 13:44:15.302767 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:15.302716 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-npd7p" podStartSLOduration=2.111498347 podStartE2EDuration="3.302700407s" totalImagesPullingTime="1.1175798s" totalInitContainerRuntime="73.62226ms" isStatefulPod=true podCreationTimestamp="2026-07-24 13:44:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:44:15.30164372 +0000 UTC m=+88.988405138" watchObservedRunningTime="2026-07-24 13:44:15.302700407 +0000 UTC m=+88.989461824" Jul 24 13:44:16.108107 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.108063 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-644fb596d7-kjvss" podUID="fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73" containerName="registry" containerID="cri-o://fdd64ae9034f9ca86f036d7ef78201ac14dc5fa2f65488b72e50944bd92964e4" gracePeriod=30 Jul 24 13:44:16.288563 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.288522 2557 generic.go:356] "Generic (PLEG): container finished" podID="fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73" containerID="fdd64ae9034f9ca86f036d7ef78201ac14dc5fa2f65488b72e50944bd92964e4" exitCode=0 Jul 24 13:44:16.289012 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.288981 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644fb596d7-kjvss" event={"ID":"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73","Type":"ContainerDied","Data":"fdd64ae9034f9ca86f036d7ef78201ac14dc5fa2f65488b72e50944bd92964e4"} Jul 24 13:44:16.355019 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.354997 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:44:16.434607 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.434524 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-certificates\") pod \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " Jul 24 13:44:16.434607 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.434562 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkpbx\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-kube-api-access-xkpbx\") pod \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " Jul 24 13:44:16.434607 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.434583 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-image-registry-private-configuration\") pod \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " Jul 24 13:44:16.434904 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.434728 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-bound-sa-token\") pod \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " Jul 24 13:44:16.434904 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.434785 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-installation-pull-secrets\") pod \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " Jul 24 13:44:16.434904 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.434844 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-ca-trust-extracted\") pod \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " Jul 24 13:44:16.434904 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.434871 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-trusted-ca\") pod \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " Jul 24 13:44:16.435103 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.434908 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls\") pod \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\" (UID: \"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73\") " Jul 24 13:44:16.435849 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.435172 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73" (UID: "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:44:16.435849 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.435586 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73" (UID: "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:44:16.437626 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.437595 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73" (UID: "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:44:16.437769 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.437741 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73" (UID: "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:44:16.437883 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.437859 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-kube-api-access-xkpbx" (OuterVolumeSpecName: "kube-api-access-xkpbx") pod "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73" (UID: "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73"). InnerVolumeSpecName "kube-api-access-xkpbx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:44:16.437956 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.437873 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73" (UID: "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:44:16.438068 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.438050 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73" (UID: "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:44:16.444098 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.444076 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73" (UID: "fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:44:16.536473 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.536433 2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-bound-sa-token\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:44:16.536473 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.536466 2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-installation-pull-secrets\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:44:16.536473 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.536479 2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-ca-trust-extracted\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:44:16.536717 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.536491 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-trusted-ca\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:44:16.536717 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.536503 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-tls\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:44:16.536717 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.536529 2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-registry-certificates\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:44:16.536717 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.536540 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkpbx\" (UniqueName: \"kubernetes.io/projected/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-kube-api-access-xkpbx\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:44:16.536717 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:16.536552 2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73-image-registry-private-configuration\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:44:17.293423 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:17.293345 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-644fb596d7-kjvss" Jul 24 13:44:17.293896 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:17.293337 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-644fb596d7-kjvss" event={"ID":"fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73","Type":"ContainerDied","Data":"311f92ae8563ca903a3f4edcbbf564238aafc0799dca2ad42e2cd109b19ad470"} Jul 24 13:44:17.293896 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:17.293478 2557 scope.go:117] "RemoveContainer" containerID="fdd64ae9034f9ca86f036d7ef78201ac14dc5fa2f65488b72e50944bd92964e4" Jul 24 13:44:17.314968 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:17.314941 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-644fb596d7-kjvss"] Jul 24 13:44:17.322289 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:17.322264 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-644fb596d7-kjvss"] Jul 24 13:44:18.876487 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:18.876455 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73" path="/var/lib/kubelet/pods/fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73/volumes" Jul 24 13:44:33.238543 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:44:33.238512 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9sfhz" Jul 24 13:47:31.119614 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.119581 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8"] Jul 24 13:47:31.120050 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.119834 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73" containerName="registry" Jul 24 13:47:31.120050 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.119844 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73" containerName="registry" Jul 24 13:47:31.120050 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.119892 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdbc1a29-cdfb-4789-abf6-f6bfbe22ce73" containerName="registry" Jul 24 13:47:31.122600 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.122585 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.124863 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.124843 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-665mt\"" Jul 24 13:47:31.124979 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.124936 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 13:47:31.125263 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.125246 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 24 13:47:31.125336 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.125254 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 13:47:31.125336 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.125282 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 13:47:31.137088 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.137059 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8"] Jul 24 13:47:31.244909 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.244876 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz4g4\" (UniqueName: \"kubernetes.io/projected/a737c1d9-5044-4cb2-a972-d206ce0c0df5-kube-api-access-sz4g4\") pod \"opendatahub-operator-controller-manager-674cffbf7-snrf8\" (UID: \"a737c1d9-5044-4cb2-a972-d206ce0c0df5\") " pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.245100 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.244917 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a737c1d9-5044-4cb2-a972-d206ce0c0df5-tmp\") pod \"opendatahub-operator-controller-manager-674cffbf7-snrf8\" (UID: \"a737c1d9-5044-4cb2-a972-d206ce0c0df5\") " pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.245100 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.244941 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a737c1d9-5044-4cb2-a972-d206ce0c0df5-webhook-cert\") pod \"opendatahub-operator-controller-manager-674cffbf7-snrf8\" (UID: \"a737c1d9-5044-4cb2-a972-d206ce0c0df5\") " pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.245100 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.245012 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a737c1d9-5044-4cb2-a972-d206ce0c0df5-manifests\") pod \"opendatahub-operator-controller-manager-674cffbf7-snrf8\" (UID: \"a737c1d9-5044-4cb2-a972-d206ce0c0df5\") " pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.245100 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.245033 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a737c1d9-5044-4cb2-a972-d206ce0c0df5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-674cffbf7-snrf8\" (UID: \"a737c1d9-5044-4cb2-a972-d206ce0c0df5\") " pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.345711 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.345676 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz4g4\" (UniqueName: \"kubernetes.io/projected/a737c1d9-5044-4cb2-a972-d206ce0c0df5-kube-api-access-sz4g4\") pod \"opendatahub-operator-controller-manager-674cffbf7-snrf8\" (UID: \"a737c1d9-5044-4cb2-a972-d206ce0c0df5\") " pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.345711 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.345712 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a737c1d9-5044-4cb2-a972-d206ce0c0df5-tmp\") pod \"opendatahub-operator-controller-manager-674cffbf7-snrf8\" (UID: \"a737c1d9-5044-4cb2-a972-d206ce0c0df5\") " pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.345970 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.345739 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a737c1d9-5044-4cb2-a972-d206ce0c0df5-webhook-cert\") pod \"opendatahub-operator-controller-manager-674cffbf7-snrf8\" (UID: \"a737c1d9-5044-4cb2-a972-d206ce0c0df5\") " pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.345970 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.345768 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a737c1d9-5044-4cb2-a972-d206ce0c0df5-manifests\") pod \"opendatahub-operator-controller-manager-674cffbf7-snrf8\" (UID: \"a737c1d9-5044-4cb2-a972-d206ce0c0df5\") " pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.345970 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.345793 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a737c1d9-5044-4cb2-a972-d206ce0c0df5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-674cffbf7-snrf8\" (UID: \"a737c1d9-5044-4cb2-a972-d206ce0c0df5\") " pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.346248 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.346224 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a737c1d9-5044-4cb2-a972-d206ce0c0df5-manifests\") pod \"opendatahub-operator-controller-manager-674cffbf7-snrf8\" (UID: \"a737c1d9-5044-4cb2-a972-d206ce0c0df5\") " pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.348248 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.348227 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a737c1d9-5044-4cb2-a972-d206ce0c0df5-tmp\") pod \"opendatahub-operator-controller-manager-674cffbf7-snrf8\" (UID: \"a737c1d9-5044-4cb2-a972-d206ce0c0df5\") " pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.348401 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.348383 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a737c1d9-5044-4cb2-a972-d206ce0c0df5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-674cffbf7-snrf8\" (UID: \"a737c1d9-5044-4cb2-a972-d206ce0c0df5\") " pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.348521 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.348424 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a737c1d9-5044-4cb2-a972-d206ce0c0df5-webhook-cert\") pod \"opendatahub-operator-controller-manager-674cffbf7-snrf8\" (UID: \"a737c1d9-5044-4cb2-a972-d206ce0c0df5\") " pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.356101 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.356069 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz4g4\" (UniqueName: \"kubernetes.io/projected/a737c1d9-5044-4cb2-a972-d206ce0c0df5-kube-api-access-sz4g4\") pod \"opendatahub-operator-controller-manager-674cffbf7-snrf8\" (UID: \"a737c1d9-5044-4cb2-a972-d206ce0c0df5\") " pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.432488 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.432403 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:31.580695 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.580663 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8"] Jul 24 13:47:31.584255 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:47:31.584229 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda737c1d9_5044_4cb2_a972_d206ce0c0df5.slice/crio-c78b787be393bb5c3d122f9d1c8e44252c134d54b54952e9818f61b3d8687412 WatchSource:0}: Error finding container c78b787be393bb5c3d122f9d1c8e44252c134d54b54952e9818f61b3d8687412: Status 404 returned error can't find the container with id c78b787be393bb5c3d122f9d1c8e44252c134d54b54952e9818f61b3d8687412 Jul 24 13:47:31.933374 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:31.933340 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" event={"ID":"a737c1d9-5044-4cb2-a972-d206ce0c0df5","Type":"ContainerStarted","Data":"c78b787be393bb5c3d122f9d1c8e44252c134d54b54952e9818f61b3d8687412"} Jul 24 13:47:34.945940 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:34.945901 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" event={"ID":"a737c1d9-5044-4cb2-a972-d206ce0c0df5","Type":"ContainerStarted","Data":"8413cde13d92a8f391b4aaf0e07e4b0f75785a7c3a961c895ca18b402f3765f1"} Jul 24 13:47:35.950431 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:35.950334 2557 generic.go:356] "Generic (PLEG): container finished" podID="a737c1d9-5044-4cb2-a972-d206ce0c0df5" containerID="8413cde13d92a8f391b4aaf0e07e4b0f75785a7c3a961c895ca18b402f3765f1" exitCode=0 Jul 24 13:47:35.950431 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:35.950379 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" event={"ID":"a737c1d9-5044-4cb2-a972-d206ce0c0df5","Type":"ContainerDied","Data":"8413cde13d92a8f391b4aaf0e07e4b0f75785a7c3a961c895ca18b402f3765f1"} Jul 24 13:47:36.955996 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:36.955958 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" event={"ID":"a737c1d9-5044-4cb2-a972-d206ce0c0df5","Type":"ContainerStarted","Data":"88f250fa4ac7e5e7fd891668a706dc56ba1d8b4ad78f4f2a1cedd398915055f4"} Jul 24 13:47:36.956390 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:36.956103 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:36.977664 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:36.977608 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" podStartSLOduration=1.67209575 podStartE2EDuration="5.977590886s" totalImagesPullingTime="3.219713302s" totalInitContainerRuntime="1.085781834s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:47:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:47:36.97597734 +0000 UTC m=+290.662738759" watchObservedRunningTime="2026-07-24 13:47:36.977590886 +0000 UTC m=+290.664352304" Jul 24 13:47:46.714697 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:46.714668 2557 kubelet.go:1648] "Image garbage collection succeeded" Jul 24 13:47:47.961757 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:47.961729 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-674cffbf7-snrf8" Jul 24 13:47:50.887101 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:50.887065 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz"] Jul 24 13:47:50.889495 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:50.889472 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz" Jul 24 13:47:50.891918 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:50.891890 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 24 13:47:50.892051 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:50.891953 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-v9fll\"" Jul 24 13:47:50.892523 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:50.892503 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 24 13:47:50.892523 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:50.892515 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 24 13:47:50.893465 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:50.893448 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 24 13:47:50.912249 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:50.912222 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz"] Jul 24 13:47:51.002294 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.002261 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f322fa1-0998-4846-b50e-436ae1949227-tmp\") pod \"kube-auth-proxy-5c9467694d-hhcmz\" (UID: \"0f322fa1-0998-4846-b50e-436ae1949227\") " pod="openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz" Jul 24 13:47:51.002294 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.002298 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ws99\" (UniqueName: \"kubernetes.io/projected/0f322fa1-0998-4846-b50e-436ae1949227-kube-api-access-9ws99\") pod \"kube-auth-proxy-5c9467694d-hhcmz\" (UID: \"0f322fa1-0998-4846-b50e-436ae1949227\") " pod="openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz" Jul 24 13:47:51.002568 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.002399 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f322fa1-0998-4846-b50e-436ae1949227-tls-certs\") pod \"kube-auth-proxy-5c9467694d-hhcmz\" (UID: \"0f322fa1-0998-4846-b50e-436ae1949227\") " pod="openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz" Jul 24 13:47:51.103800 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.103755 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f322fa1-0998-4846-b50e-436ae1949227-tmp\") pod \"kube-auth-proxy-5c9467694d-hhcmz\" (UID: \"0f322fa1-0998-4846-b50e-436ae1949227\") " pod="openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz" Jul 24 13:47:51.103800 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.103793 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ws99\" (UniqueName: \"kubernetes.io/projected/0f322fa1-0998-4846-b50e-436ae1949227-kube-api-access-9ws99\") pod \"kube-auth-proxy-5c9467694d-hhcmz\" (UID: \"0f322fa1-0998-4846-b50e-436ae1949227\") " pod="openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz" Jul 24 13:47:51.104082 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.103901 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f322fa1-0998-4846-b50e-436ae1949227-tls-certs\") pod \"kube-auth-proxy-5c9467694d-hhcmz\" (UID: \"0f322fa1-0998-4846-b50e-436ae1949227\") " pod="openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz" Jul 24 13:47:51.106344 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.106312 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f322fa1-0998-4846-b50e-436ae1949227-tmp\") pod \"kube-auth-proxy-5c9467694d-hhcmz\" (UID: \"0f322fa1-0998-4846-b50e-436ae1949227\") " pod="openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz" Jul 24 13:47:51.106619 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.106595 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f322fa1-0998-4846-b50e-436ae1949227-tls-certs\") pod \"kube-auth-proxy-5c9467694d-hhcmz\" (UID: \"0f322fa1-0998-4846-b50e-436ae1949227\") " pod="openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz" Jul 24 13:47:51.137586 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.137504 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6fftq"] Jul 24 13:47:51.139088 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.139065 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ws99\" (UniqueName: \"kubernetes.io/projected/0f322fa1-0998-4846-b50e-436ae1949227-kube-api-access-9ws99\") pod \"kube-auth-proxy-5c9467694d-hhcmz\" (UID: \"0f322fa1-0998-4846-b50e-436ae1949227\") " pod="openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz" Jul 24 13:47:51.139582 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.139568 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6fftq" Jul 24 13:47:51.142268 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.142252 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 24 13:47:51.143058 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.143044 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-2frxs\"" Jul 24 13:47:51.202484 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.202456 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz" Jul 24 13:47:51.204469 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.204444 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb84f68e-d4ae-43c2-bc54-33c496c21c71-cert\") pod \"odh-model-controller-7998c67dc4-6fftq\" (UID: \"bb84f68e-d4ae-43c2-bc54-33c496c21c71\") " pod="opendatahub/odh-model-controller-7998c67dc4-6fftq" Jul 24 13:47:51.204592 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.204492 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb55h\" (UniqueName: \"kubernetes.io/projected/bb84f68e-d4ae-43c2-bc54-33c496c21c71-kube-api-access-zb55h\") pod \"odh-model-controller-7998c67dc4-6fftq\" (UID: \"bb84f68e-d4ae-43c2-bc54-33c496c21c71\") " pod="opendatahub/odh-model-controller-7998c67dc4-6fftq" Jul 24 13:47:51.295259 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.295225 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6fftq"] Jul 24 13:47:51.305862 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.305828 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb84f68e-d4ae-43c2-bc54-33c496c21c71-cert\") pod \"odh-model-controller-7998c67dc4-6fftq\" (UID: \"bb84f68e-d4ae-43c2-bc54-33c496c21c71\") " pod="opendatahub/odh-model-controller-7998c67dc4-6fftq" Jul 24 13:47:51.306035 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.305884 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb55h\" (UniqueName: \"kubernetes.io/projected/bb84f68e-d4ae-43c2-bc54-33c496c21c71-kube-api-access-zb55h\") pod \"odh-model-controller-7998c67dc4-6fftq\" (UID: \"bb84f68e-d4ae-43c2-bc54-33c496c21c71\") " pod="opendatahub/odh-model-controller-7998c67dc4-6fftq" Jul 24 13:47:51.306337 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:47:51.306315 2557 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 24 13:47:51.306421 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:47:51.306396 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb84f68e-d4ae-43c2-bc54-33c496c21c71-cert podName:bb84f68e-d4ae-43c2-bc54-33c496c21c71 nodeName:}" failed. No retries permitted until 2026-07-24 13:47:51.80637469 +0000 UTC m=+305.493136108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bb84f68e-d4ae-43c2-bc54-33c496c21c71-cert") pod "odh-model-controller-7998c67dc4-6fftq" (UID: "bb84f68e-d4ae-43c2-bc54-33c496c21c71") : secret "odh-model-controller-webhook-cert" not found Jul 24 13:47:51.356125 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.356095 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb55h\" (UniqueName: \"kubernetes.io/projected/bb84f68e-d4ae-43c2-bc54-33c496c21c71-kube-api-access-zb55h\") pod \"odh-model-controller-7998c67dc4-6fftq\" (UID: \"bb84f68e-d4ae-43c2-bc54-33c496c21c71\") " pod="opendatahub/odh-model-controller-7998c67dc4-6fftq" Jul 24 13:47:51.376434 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.376402 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz"] Jul 24 13:47:51.376702 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:47:51.376681 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f322fa1_0998_4846_b50e_436ae1949227.slice/crio-0d4cd9356c3fb05cbcbbe90c7fc24f5cfd83e3fae2908b71d001c6b441841457 WatchSource:0}: Error finding container 0d4cd9356c3fb05cbcbbe90c7fc24f5cfd83e3fae2908b71d001c6b441841457: Status 404 returned error can't find the container with id 0d4cd9356c3fb05cbcbbe90c7fc24f5cfd83e3fae2908b71d001c6b441841457 Jul 24 13:47:51.379123 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.379105 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:47:51.812152 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.812112 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb84f68e-d4ae-43c2-bc54-33c496c21c71-cert\") pod \"odh-model-controller-7998c67dc4-6fftq\" (UID: \"bb84f68e-d4ae-43c2-bc54-33c496c21c71\") " pod="opendatahub/odh-model-controller-7998c67dc4-6fftq" Jul 24 13:47:51.814865 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:51.814838 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb84f68e-d4ae-43c2-bc54-33c496c21c71-cert\") pod \"odh-model-controller-7998c67dc4-6fftq\" (UID: \"bb84f68e-d4ae-43c2-bc54-33c496c21c71\") " pod="opendatahub/odh-model-controller-7998c67dc4-6fftq" Jul 24 13:47:52.009589 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:52.009549 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz" event={"ID":"0f322fa1-0998-4846-b50e-436ae1949227","Type":"ContainerStarted","Data":"0d4cd9356c3fb05cbcbbe90c7fc24f5cfd83e3fae2908b71d001c6b441841457"} Jul 24 13:47:52.053554 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:52.053519 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6fftq" Jul 24 13:47:52.241763 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:52.241732 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6fftq"] Jul 24 13:47:52.244933 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:47:52.244886 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb84f68e_d4ae_43c2_bc54_33c496c21c71.slice/crio-b0d9f626a980c9c923eeb7c8038cd481a01b8c7ed55de85cfa012550782d03ed WatchSource:0}: Error finding container b0d9f626a980c9c923eeb7c8038cd481a01b8c7ed55de85cfa012550782d03ed: Status 404 returned error can't find the container with id b0d9f626a980c9c923eeb7c8038cd481a01b8c7ed55de85cfa012550782d03ed Jul 24 13:47:53.018190 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:53.018110 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6fftq" event={"ID":"bb84f68e-d4ae-43c2-bc54-33c496c21c71","Type":"ContainerStarted","Data":"b0d9f626a980c9c923eeb7c8038cd481a01b8c7ed55de85cfa012550782d03ed"} Jul 24 13:47:56.032702 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:56.032655 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6fftq" event={"ID":"bb84f68e-d4ae-43c2-bc54-33c496c21c71","Type":"ContainerStarted","Data":"82ab257e8d05e34206a802c89d64aafeacd8cc0027a718fc1bb801e7ca95463f"} Jul 24 13:47:56.033220 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:56.032747 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-6fftq" Jul 24 13:47:56.034097 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:56.034073 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz" event={"ID":"0f322fa1-0998-4846-b50e-436ae1949227","Type":"ContainerStarted","Data":"5f7d2cfb156628c3739379a72e21dc2f49abc86850e34c0fc1f4b1bef4348050"} Jul 24 13:47:56.049889 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:56.049833 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-6fftq" podStartSLOduration=1.9389829710000002 podStartE2EDuration="5.04979202s" totalImagesPullingTime="3.110809049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:47:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:47:56.048990268 +0000 UTC m=+309.735751685" watchObservedRunningTime="2026-07-24 13:47:56.04979202 +0000 UTC m=+309.736553439" Jul 24 13:47:56.065711 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:56.065651 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5c9467694d-hhcmz" podStartSLOduration=2.135849481 podStartE2EDuration="6.065631371s" totalImagesPullingTime="3.92978189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:47:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:47:56.064176877 +0000 UTC m=+309.750938295" watchObservedRunningTime="2026-07-24 13:47:56.065631371 +0000 UTC m=+309.752392786" Jul 24 13:47:58.021307 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.021272 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z"] Jul 24 13:47:58.024648 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.024627 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.027582 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.027561 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-gg8kr\"" Jul 24 13:47:58.027832 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.027796 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 13:47:58.057531 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.057500 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z"] Jul 24 13:47:58.065914 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.065878 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d222f02-d311-4503-be99-7c1ec3043ac5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.065914 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.065917 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d222f02-d311-4503-be99-7c1ec3043ac5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.066154 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.065946 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d222f02-d311-4503-be99-7c1ec3043ac5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.066154 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.066053 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqmv5\" (UniqueName: \"kubernetes.io/projected/8d222f02-d311-4503-be99-7c1ec3043ac5-kube-api-access-lqmv5\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.066154 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.066099 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d222f02-d311-4503-be99-7c1ec3043ac5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.066154 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.066149 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d222f02-d311-4503-be99-7c1ec3043ac5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.066357 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.066178 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d222f02-d311-4503-be99-7c1ec3043ac5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.066357 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.066201 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d222f02-d311-4503-be99-7c1ec3043ac5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.066357 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.066221 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d222f02-d311-4503-be99-7c1ec3043ac5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.167323 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.167289 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d222f02-d311-4503-be99-7c1ec3043ac5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.167323 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.167329 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d222f02-d311-4503-be99-7c1ec3043ac5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.167590 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.167350 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d222f02-d311-4503-be99-7c1ec3043ac5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.167590 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.167388 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqmv5\" (UniqueName: \"kubernetes.io/projected/8d222f02-d311-4503-be99-7c1ec3043ac5-kube-api-access-lqmv5\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.167590 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.167420 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d222f02-d311-4503-be99-7c1ec3043ac5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.167590 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.167454 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d222f02-d311-4503-be99-7c1ec3043ac5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.167590 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.167481 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d222f02-d311-4503-be99-7c1ec3043ac5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.167590 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.167510 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d222f02-d311-4503-be99-7c1ec3043ac5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.167590 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.167534 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d222f02-d311-4503-be99-7c1ec3043ac5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.167986 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.167802 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d222f02-d311-4503-be99-7c1ec3043ac5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.167986 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.167858 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d222f02-d311-4503-be99-7c1ec3043ac5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.167986 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.167921 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d222f02-d311-4503-be99-7c1ec3043ac5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.168137 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.168006 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d222f02-d311-4503-be99-7c1ec3043ac5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.168427 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.168409 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d222f02-d311-4503-be99-7c1ec3043ac5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.169868 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.169837 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d222f02-d311-4503-be99-7c1ec3043ac5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.170048 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.170028 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d222f02-d311-4503-be99-7c1ec3043ac5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.178016 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.177993 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d222f02-d311-4503-be99-7c1ec3043ac5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.178133 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.178101 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqmv5\" (UniqueName: \"kubernetes.io/projected/8d222f02-d311-4503-be99-7c1ec3043ac5-kube-api-access-lqmv5\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z\" (UID: \"8d222f02-d311-4503-be99-7c1ec3043ac5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.335386 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.335304 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:47:58.490205 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:58.490172 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z"] Jul 24 13:47:58.494297 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:47:58.494268 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d222f02_d311_4503_be99_7c1ec3043ac5.slice/crio-c973b78a006f4ed655a69ec087c3c0ea8ea208b10812082e71dbc9428ee66a0e WatchSource:0}: Error finding container c973b78a006f4ed655a69ec087c3c0ea8ea208b10812082e71dbc9428ee66a0e: Status 404 returned error can't find the container with id c973b78a006f4ed655a69ec087c3c0ea8ea208b10812082e71dbc9428ee66a0e Jul 24 13:47:59.047043 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.047007 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" event={"ID":"8d222f02-d311-4503-be99-7c1ec3043ac5","Type":"ContainerStarted","Data":"c973b78a006f4ed655a69ec087c3c0ea8ea208b10812082e71dbc9428ee66a0e"} Jul 24 13:47:59.146914 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.146877 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5"] Jul 24 13:47:59.151273 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.151251 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" Jul 24 13:47:59.157274 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.157253 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 24 13:47:59.157855 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.157839 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 24 13:47:59.158494 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.158470 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-nbxhg\"" Jul 24 13:47:59.158494 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.158477 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 24 13:47:59.158784 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.158766 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 24 13:47:59.159461 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.159446 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:47:59.176472 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.176449 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54-metrics-cert\") pod \"lws-controller-manager-78d5cc8c46-pvcx5\" (UID: \"c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54\") " pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" Jul 24 13:47:59.176595 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.176495 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54-manager-config\") pod \"lws-controller-manager-78d5cc8c46-pvcx5\" (UID: \"c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54\") " pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" Jul 24 13:47:59.176595 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.176516 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf22s\" (UniqueName: \"kubernetes.io/projected/c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54-kube-api-access-xf22s\") pod \"lws-controller-manager-78d5cc8c46-pvcx5\" (UID: \"c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54\") " pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" Jul 24 13:47:59.176595 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.176540 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54-cert\") pod \"lws-controller-manager-78d5cc8c46-pvcx5\" (UID: \"c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54\") " pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" Jul 24 13:47:59.185036 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.185010 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5"] Jul 24 13:47:59.277910 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.277870 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54-metrics-cert\") pod \"lws-controller-manager-78d5cc8c46-pvcx5\" (UID: \"c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54\") " pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" Jul 24 13:47:59.278102 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.277943 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54-manager-config\") pod \"lws-controller-manager-78d5cc8c46-pvcx5\" (UID: \"c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54\") " pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" Jul 24 13:47:59.278102 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.277979 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf22s\" (UniqueName: \"kubernetes.io/projected/c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54-kube-api-access-xf22s\") pod \"lws-controller-manager-78d5cc8c46-pvcx5\" (UID: \"c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54\") " pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" Jul 24 13:47:59.278102 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.278017 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54-cert\") pod \"lws-controller-manager-78d5cc8c46-pvcx5\" (UID: \"c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54\") " pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" Jul 24 13:47:59.279015 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.278988 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54-manager-config\") pod \"lws-controller-manager-78d5cc8c46-pvcx5\" (UID: \"c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54\") " pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" Jul 24 13:47:59.280472 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.280453 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54-metrics-cert\") pod \"lws-controller-manager-78d5cc8c46-pvcx5\" (UID: \"c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54\") " pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" Jul 24 13:47:59.280659 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.280638 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54-cert\") pod \"lws-controller-manager-78d5cc8c46-pvcx5\" (UID: \"c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54\") " pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" Jul 24 13:47:59.286518 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.286490 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf22s\" (UniqueName: \"kubernetes.io/projected/c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54-kube-api-access-xf22s\") pod \"lws-controller-manager-78d5cc8c46-pvcx5\" (UID: \"c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54\") " pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" Jul 24 13:47:59.460633 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.460592 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" Jul 24 13:47:59.601661 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:47:59.601633 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5"] Jul 24 13:47:59.603258 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:47:59.603233 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc87ff2ec_0cf3_4aa1_8e2a_cc51176a3d54.slice/crio-2323210ff1dfe00ab446fe040c5df142fd22dd06bcde6a9e04308c89ab0a8d1a WatchSource:0}: Error finding container 2323210ff1dfe00ab446fe040c5df142fd22dd06bcde6a9e04308c89ab0a8d1a: Status 404 returned error can't find the container with id 2323210ff1dfe00ab446fe040c5df142fd22dd06bcde6a9e04308c89ab0a8d1a Jul 24 13:48:00.066295 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:48:00.066254 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" event={"ID":"c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54","Type":"ContainerStarted","Data":"2323210ff1dfe00ab446fe040c5df142fd22dd06bcde6a9e04308c89ab0a8d1a"} Jul 24 13:48:03.134442 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:48:03.134385 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:48:03.134742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:48:03.134489 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:48:03.134742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:48:03.134575 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:48:04.090955 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:48:04.090916 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" event={"ID":"8d222f02-d311-4503-be99-7c1ec3043ac5","Type":"ContainerStarted","Data":"5ff1465a38597e34ccdfc9066b0df88af0bbd0264f2058b48c5ff090eedcdac9"} Jul 24 13:48:04.092334 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:48:04.092314 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" event={"ID":"c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54","Type":"ContainerStarted","Data":"aaf11316ab0406649b0d0e69c0b0375da9e1b8f6368f1e3f0ea8167d52411348"} Jul 24 13:48:04.092449 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:48:04.092428 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" Jul 24 13:48:04.142616 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:48:04.142563 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" podStartSLOduration=2.505370293 podStartE2EDuration="7.142546747s" totalImagesPullingTime="4.637176454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:47:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:48:04.140063569 +0000 UTC m=+317.826824986" watchObservedRunningTime="2026-07-24 13:48:04.142546747 +0000 UTC m=+317.829308165" Jul 24 13:48:04.163542 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:48:04.163496 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" podStartSLOduration=1.580891801 podStartE2EDuration="5.16348048s" totalImagesPullingTime="3.582588679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:47:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:48:04.162428315 +0000 UTC m=+317.849189743" watchObservedRunningTime="2026-07-24 13:48:04.16348048 +0000 UTC m=+317.850241896" Jul 24 13:48:04.335546 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:48:04.335509 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:48:04.340486 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:48:04.340458 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:48:05.096057 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:48:05.096023 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:48:05.097069 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:48:05.097053 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z" Jul 24 13:48:07.040128 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:48:07.040097 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-6fftq" Jul 24 13:48:15.098268 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:48:15.098238 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78d5cc8c46-pvcx5" Jul 24 13:50:46.054354 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.054320 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-bczc8"] Jul 24 13:50:46.057608 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.057591 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-bczc8" Jul 24 13:50:46.059768 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.059751 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 13:50:46.060343 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.060325 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 13:50:46.060395 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.060378 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bn8tr\"" Jul 24 13:50:46.064262 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.064236 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-bczc8"] Jul 24 13:50:46.152828 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.152765 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-dtkts"] Jul 24 13:50:46.155988 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.155966 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dtkts" Jul 24 13:50:46.163564 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.163539 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dtkts"] Jul 24 13:50:46.202137 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.202106 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2w67\" (UniqueName: \"kubernetes.io/projected/4078f2e5-680b-403c-8d43-12d4aed18a43-kube-api-access-w2w67\") pod \"authorino-58cd955675-bczc8\" (UID: \"4078f2e5-680b-403c-8d43-12d4aed18a43\") " pod="kuadrant-system/authorino-58cd955675-bczc8" Jul 24 13:50:46.302983 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.302949 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxflw\" (UniqueName: \"kubernetes.io/projected/8ed75a6c-c13d-4524-b2cb-978fe5781631-kube-api-access-kxflw\") pod \"authorino-75d556bd77-dtkts\" (UID: \"8ed75a6c-c13d-4524-b2cb-978fe5781631\") " pod="kuadrant-system/authorino-75d556bd77-dtkts" Jul 24 13:50:46.303169 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.303003 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2w67\" (UniqueName: \"kubernetes.io/projected/4078f2e5-680b-403c-8d43-12d4aed18a43-kube-api-access-w2w67\") pod \"authorino-58cd955675-bczc8\" (UID: \"4078f2e5-680b-403c-8d43-12d4aed18a43\") " pod="kuadrant-system/authorino-58cd955675-bczc8" Jul 24 13:50:46.311295 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.311238 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2w67\" (UniqueName: \"kubernetes.io/projected/4078f2e5-680b-403c-8d43-12d4aed18a43-kube-api-access-w2w67\") pod \"authorino-58cd955675-bczc8\" (UID: \"4078f2e5-680b-403c-8d43-12d4aed18a43\") " pod="kuadrant-system/authorino-58cd955675-bczc8" Jul 24 13:50:46.367873 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.367839 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-bczc8" Jul 24 13:50:46.404272 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.404238 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxflw\" (UniqueName: \"kubernetes.io/projected/8ed75a6c-c13d-4524-b2cb-978fe5781631-kube-api-access-kxflw\") pod \"authorino-75d556bd77-dtkts\" (UID: \"8ed75a6c-c13d-4524-b2cb-978fe5781631\") " pod="kuadrant-system/authorino-75d556bd77-dtkts" Jul 24 13:50:46.418587 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.418516 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxflw\" (UniqueName: \"kubernetes.io/projected/8ed75a6c-c13d-4524-b2cb-978fe5781631-kube-api-access-kxflw\") pod \"authorino-75d556bd77-dtkts\" (UID: \"8ed75a6c-c13d-4524-b2cb-978fe5781631\") " pod="kuadrant-system/authorino-75d556bd77-dtkts" Jul 24 13:50:46.465143 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.465112 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dtkts" Jul 24 13:50:46.499689 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.499609 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-bczc8"] Jul 24 13:50:46.502354 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:50:46.502323 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4078f2e5_680b_403c_8d43_12d4aed18a43.slice/crio-e1120397d21bd8ebdcdc8a6442e366420ab43cf26a76fbb65cd6ecfeeb5a0c3e WatchSource:0}: Error finding container e1120397d21bd8ebdcdc8a6442e366420ab43cf26a76fbb65cd6ecfeeb5a0c3e: Status 404 returned error can't find the container with id e1120397d21bd8ebdcdc8a6442e366420ab43cf26a76fbb65cd6ecfeeb5a0c3e Jul 24 13:50:46.597075 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.597052 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dtkts"] Jul 24 13:50:46.599500 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:50:46.599470 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ed75a6c_c13d_4524_b2cb_978fe5781631.slice/crio-42858311d25a9db2086557d18a78c2fb3ffc9c2dee5b41fe510fafa301915bf0 WatchSource:0}: Error finding container 42858311d25a9db2086557d18a78c2fb3ffc9c2dee5b41fe510fafa301915bf0: Status 404 returned error can't find the container with id 42858311d25a9db2086557d18a78c2fb3ffc9c2dee5b41fe510fafa301915bf0 Jul 24 13:50:46.732913 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.732881 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dtkts" event={"ID":"8ed75a6c-c13d-4524-b2cb-978fe5781631","Type":"ContainerStarted","Data":"42858311d25a9db2086557d18a78c2fb3ffc9c2dee5b41fe510fafa301915bf0"} Jul 24 13:50:46.734022 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:46.733999 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-bczc8" event={"ID":"4078f2e5-680b-403c-8d43-12d4aed18a43","Type":"ContainerStarted","Data":"e1120397d21bd8ebdcdc8a6442e366420ab43cf26a76fbb65cd6ecfeeb5a0c3e"} Jul 24 13:50:50.758445 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:50.758407 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dtkts" event={"ID":"8ed75a6c-c13d-4524-b2cb-978fe5781631","Type":"ContainerStarted","Data":"4c90084817d579486808f889f6ffbd4f87ad7b3c4b67b91a8d593d40b6660da7"} Jul 24 13:50:50.759917 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:50.759894 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-bczc8" event={"ID":"4078f2e5-680b-403c-8d43-12d4aed18a43","Type":"ContainerStarted","Data":"bf19c09020c47cdf45e46ad439f96104423a7089d43af0828d42bbc8c76d019d"} Jul 24 13:50:50.772626 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:50.772575 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-dtkts" podStartSLOduration=1.373097961 podStartE2EDuration="4.772559436s" totalImagesPullingTime="3.399461475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:50:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:50:50.772339937 +0000 UTC m=+484.459101357" watchObservedRunningTime="2026-07-24 13:50:50.772559436 +0000 UTC m=+484.459320853" Jul 24 13:50:50.791182 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:50.791122 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-bczc8" podStartSLOduration=1.304951046 podStartE2EDuration="4.791099559s" totalImagesPullingTime="3.486148513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:50:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:50:50.79022546 +0000 UTC m=+484.476986878" watchObservedRunningTime="2026-07-24 13:50:50.791099559 +0000 UTC m=+484.477860977" Jul 24 13:50:50.799633 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:50.799603 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-bczc8"] Jul 24 13:50:52.767647 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:52.767611 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-bczc8" podUID="4078f2e5-680b-403c-8d43-12d4aed18a43" containerName="authorino" containerID="cri-o://bf19c09020c47cdf45e46ad439f96104423a7089d43af0828d42bbc8c76d019d" gracePeriod=30 Jul 24 13:50:53.012374 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:53.012349 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-bczc8" Jul 24 13:50:53.168910 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:53.168878 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2w67\" (UniqueName: \"kubernetes.io/projected/4078f2e5-680b-403c-8d43-12d4aed18a43-kube-api-access-w2w67\") pod \"4078f2e5-680b-403c-8d43-12d4aed18a43\" (UID: \"4078f2e5-680b-403c-8d43-12d4aed18a43\") " Jul 24 13:50:53.171254 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:53.171221 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4078f2e5-680b-403c-8d43-12d4aed18a43-kube-api-access-w2w67" (OuterVolumeSpecName: "kube-api-access-w2w67") pod "4078f2e5-680b-403c-8d43-12d4aed18a43" (UID: "4078f2e5-680b-403c-8d43-12d4aed18a43"). InnerVolumeSpecName "kube-api-access-w2w67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:50:53.269713 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:53.269670 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2w67\" (UniqueName: \"kubernetes.io/projected/4078f2e5-680b-403c-8d43-12d4aed18a43-kube-api-access-w2w67\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:50:53.772365 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:53.772331 2557 generic.go:356] "Generic (PLEG): container finished" podID="4078f2e5-680b-403c-8d43-12d4aed18a43" containerID="bf19c09020c47cdf45e46ad439f96104423a7089d43af0828d42bbc8c76d019d" exitCode=0 Jul 24 13:50:53.772748 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:53.772374 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-bczc8" Jul 24 13:50:53.772748 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:53.772413 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-bczc8" event={"ID":"4078f2e5-680b-403c-8d43-12d4aed18a43","Type":"ContainerDied","Data":"bf19c09020c47cdf45e46ad439f96104423a7089d43af0828d42bbc8c76d019d"} Jul 24 13:50:53.772748 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:53.772453 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-bczc8" event={"ID":"4078f2e5-680b-403c-8d43-12d4aed18a43","Type":"ContainerDied","Data":"e1120397d21bd8ebdcdc8a6442e366420ab43cf26a76fbb65cd6ecfeeb5a0c3e"} Jul 24 13:50:53.772748 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:53.772476 2557 scope.go:117] "RemoveContainer" containerID="bf19c09020c47cdf45e46ad439f96104423a7089d43af0828d42bbc8c76d019d" Jul 24 13:50:53.780799 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:53.780781 2557 scope.go:117] "RemoveContainer" containerID="bf19c09020c47cdf45e46ad439f96104423a7089d43af0828d42bbc8c76d019d" Jul 24 13:50:53.781104 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:50:53.781086 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf19c09020c47cdf45e46ad439f96104423a7089d43af0828d42bbc8c76d019d\": container with ID starting with bf19c09020c47cdf45e46ad439f96104423a7089d43af0828d42bbc8c76d019d not found: ID does not exist" containerID="bf19c09020c47cdf45e46ad439f96104423a7089d43af0828d42bbc8c76d019d" Jul 24 13:50:53.781148 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:53.781114 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf19c09020c47cdf45e46ad439f96104423a7089d43af0828d42bbc8c76d019d"} err="failed to get container status \"bf19c09020c47cdf45e46ad439f96104423a7089d43af0828d42bbc8c76d019d\": rpc error: code = NotFound desc = could not find container \"bf19c09020c47cdf45e46ad439f96104423a7089d43af0828d42bbc8c76d019d\": container with ID starting with bf19c09020c47cdf45e46ad439f96104423a7089d43af0828d42bbc8c76d019d not found: ID does not exist" Jul 24 13:50:53.793098 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:53.793070 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-bczc8"] Jul 24 13:50:53.797101 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:53.797078 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-bczc8"] Jul 24 13:50:54.877533 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:50:54.877494 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4078f2e5-680b-403c-8d43-12d4aed18a43" path="/var/lib/kubelet/pods/4078f2e5-680b-403c-8d43-12d4aed18a43/volumes" Jul 24 13:51:15.968235 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:15.968190 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-54czq"] Jul 24 13:51:15.968722 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:15.968472 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4078f2e5-680b-403c-8d43-12d4aed18a43" containerName="authorino" Jul 24 13:51:15.968722 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:15.968483 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4078f2e5-680b-403c-8d43-12d4aed18a43" containerName="authorino" Jul 24 13:51:15.968722 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:15.968538 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4078f2e5-680b-403c-8d43-12d4aed18a43" containerName="authorino" Jul 24 13:51:15.971753 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:15.971735 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-54czq" Jul 24 13:51:15.978074 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:15.978047 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-54czq"] Jul 24 13:51:16.038870 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:16.038802 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4shs\" (UniqueName: \"kubernetes.io/projected/7bcee36d-c07d-4a31-a5f6-6f110c087030-kube-api-access-t4shs\") pod \"authorino-5d4b67c446-54czq\" (UID: \"7bcee36d-c07d-4a31-a5f6-6f110c087030\") " pod="kuadrant-system/authorino-5d4b67c446-54czq" Jul 24 13:51:16.139895 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:16.139796 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4shs\" (UniqueName: \"kubernetes.io/projected/7bcee36d-c07d-4a31-a5f6-6f110c087030-kube-api-access-t4shs\") pod \"authorino-5d4b67c446-54czq\" (UID: \"7bcee36d-c07d-4a31-a5f6-6f110c087030\") " pod="kuadrant-system/authorino-5d4b67c446-54czq" Jul 24 13:51:16.148447 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:16.148420 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4shs\" (UniqueName: \"kubernetes.io/projected/7bcee36d-c07d-4a31-a5f6-6f110c087030-kube-api-access-t4shs\") pod \"authorino-5d4b67c446-54czq\" (UID: \"7bcee36d-c07d-4a31-a5f6-6f110c087030\") " pod="kuadrant-system/authorino-5d4b67c446-54czq" Jul 24 13:51:16.258115 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:16.258033 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-54czq"] Jul 24 13:51:16.258325 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:16.258314 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-54czq" Jul 24 13:51:16.388506 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:16.388482 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-54czq"] Jul 24 13:51:16.390992 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:51:16.390970 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bcee36d_c07d_4a31_a5f6_6f110c087030.slice/crio-7c0ddab90a4cd06cd8672bcc997193d95da16435c878ce1abc29e5645dad3fc7 WatchSource:0}: Error finding container 7c0ddab90a4cd06cd8672bcc997193d95da16435c878ce1abc29e5645dad3fc7: Status 404 returned error can't find the container with id 7c0ddab90a4cd06cd8672bcc997193d95da16435c878ce1abc29e5645dad3fc7 Jul 24 13:51:16.876936 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:16.876901 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-54czq" podUID="7bcee36d-c07d-4a31-a5f6-6f110c087030" containerName="authorino" containerID="cri-o://54e6d9762179f187842166a392c76d744b7e7809c7eae8284e7642dd620a72d6" gracePeriod=30 Jul 24 13:51:16.877144 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:16.877069 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-54czq" event={"ID":"7bcee36d-c07d-4a31-a5f6-6f110c087030","Type":"ContainerStarted","Data":"54e6d9762179f187842166a392c76d744b7e7809c7eae8284e7642dd620a72d6"} Jul 24 13:51:16.877144 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:16.877104 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-54czq" event={"ID":"7bcee36d-c07d-4a31-a5f6-6f110c087030","Type":"ContainerStarted","Data":"7c0ddab90a4cd06cd8672bcc997193d95da16435c878ce1abc29e5645dad3fc7"} Jul 24 13:51:16.892645 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:16.892475 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-54czq" podStartSLOduration=1.569385708 podStartE2EDuration="1.892449596s" totalImagesPullingTime="323.063888ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:51:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:51:16.892423818 +0000 UTC m=+510.579185236" watchObservedRunningTime="2026-07-24 13:51:16.892449596 +0000 UTC m=+510.579211012" Jul 24 13:51:17.109060 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:17.109037 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-54czq" Jul 24 13:51:17.147143 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:17.147115 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4shs\" (UniqueName: \"kubernetes.io/projected/7bcee36d-c07d-4a31-a5f6-6f110c087030-kube-api-access-t4shs\") pod \"7bcee36d-c07d-4a31-a5f6-6f110c087030\" (UID: \"7bcee36d-c07d-4a31-a5f6-6f110c087030\") " Jul 24 13:51:17.149270 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:17.149242 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bcee36d-c07d-4a31-a5f6-6f110c087030-kube-api-access-t4shs" (OuterVolumeSpecName: "kube-api-access-t4shs") pod "7bcee36d-c07d-4a31-a5f6-6f110c087030" (UID: "7bcee36d-c07d-4a31-a5f6-6f110c087030"). InnerVolumeSpecName "kube-api-access-t4shs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:51:17.248043 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:17.247965 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4shs\" (UniqueName: \"kubernetes.io/projected/7bcee36d-c07d-4a31-a5f6-6f110c087030-kube-api-access-t4shs\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:51:17.882199 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:17.882163 2557 generic.go:356] "Generic (PLEG): container finished" podID="7bcee36d-c07d-4a31-a5f6-6f110c087030" containerID="54e6d9762179f187842166a392c76d744b7e7809c7eae8284e7642dd620a72d6" exitCode=0 Jul 24 13:51:17.882199 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:17.882203 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-54czq" Jul 24 13:51:17.882406 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:17.882251 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-54czq" event={"ID":"7bcee36d-c07d-4a31-a5f6-6f110c087030","Type":"ContainerDied","Data":"54e6d9762179f187842166a392c76d744b7e7809c7eae8284e7642dd620a72d6"} Jul 24 13:51:17.882406 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:17.882292 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-54czq" event={"ID":"7bcee36d-c07d-4a31-a5f6-6f110c087030","Type":"ContainerDied","Data":"7c0ddab90a4cd06cd8672bcc997193d95da16435c878ce1abc29e5645dad3fc7"} Jul 24 13:51:17.882406 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:17.882309 2557 scope.go:117] "RemoveContainer" containerID="54e6d9762179f187842166a392c76d744b7e7809c7eae8284e7642dd620a72d6" Jul 24 13:51:17.891263 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:17.891248 2557 scope.go:117] "RemoveContainer" containerID="54e6d9762179f187842166a392c76d744b7e7809c7eae8284e7642dd620a72d6" Jul 24 13:51:17.891528 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:51:17.891511 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54e6d9762179f187842166a392c76d744b7e7809c7eae8284e7642dd620a72d6\": container with ID starting with 54e6d9762179f187842166a392c76d744b7e7809c7eae8284e7642dd620a72d6 not found: ID does not exist" containerID="54e6d9762179f187842166a392c76d744b7e7809c7eae8284e7642dd620a72d6" Jul 24 13:51:17.891594 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:17.891537 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54e6d9762179f187842166a392c76d744b7e7809c7eae8284e7642dd620a72d6"} err="failed to get container status \"54e6d9762179f187842166a392c76d744b7e7809c7eae8284e7642dd620a72d6\": rpc error: code = NotFound desc = could not find container \"54e6d9762179f187842166a392c76d744b7e7809c7eae8284e7642dd620a72d6\": container with ID starting with 54e6d9762179f187842166a392c76d744b7e7809c7eae8284e7642dd620a72d6 not found: ID does not exist" Jul 24 13:51:17.901152 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:17.901128 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-54czq"] Jul 24 13:51:17.904665 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:17.904643 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-54czq"] Jul 24 13:51:18.876873 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:18.876841 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bcee36d-c07d-4a31-a5f6-6f110c087030" path="/var/lib/kubelet/pods/7bcee36d-c07d-4a31-a5f6-6f110c087030/volumes" Jul 24 13:51:19.611402 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:19.611370 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dtkts"] Jul 24 13:51:19.611584 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:19.611562 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-dtkts" podUID="8ed75a6c-c13d-4524-b2cb-978fe5781631" containerName="authorino" containerID="cri-o://4c90084817d579486808f889f6ffbd4f87ad7b3c4b67b91a8d593d40b6660da7" gracePeriod=30 Jul 24 13:51:19.857034 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:19.856998 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dtkts" Jul 24 13:51:19.867236 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:19.867175 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxflw\" (UniqueName: \"kubernetes.io/projected/8ed75a6c-c13d-4524-b2cb-978fe5781631-kube-api-access-kxflw\") pod \"8ed75a6c-c13d-4524-b2cb-978fe5781631\" (UID: \"8ed75a6c-c13d-4524-b2cb-978fe5781631\") " Jul 24 13:51:19.869362 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:19.869337 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ed75a6c-c13d-4524-b2cb-978fe5781631-kube-api-access-kxflw" (OuterVolumeSpecName: "kube-api-access-kxflw") pod "8ed75a6c-c13d-4524-b2cb-978fe5781631" (UID: "8ed75a6c-c13d-4524-b2cb-978fe5781631"). InnerVolumeSpecName "kube-api-access-kxflw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:51:19.892503 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:19.892470 2557 generic.go:356] "Generic (PLEG): container finished" podID="8ed75a6c-c13d-4524-b2cb-978fe5781631" containerID="4c90084817d579486808f889f6ffbd4f87ad7b3c4b67b91a8d593d40b6660da7" exitCode=0 Jul 24 13:51:19.892881 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:19.892522 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dtkts" Jul 24 13:51:19.892881 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:19.892541 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dtkts" event={"ID":"8ed75a6c-c13d-4524-b2cb-978fe5781631","Type":"ContainerDied","Data":"4c90084817d579486808f889f6ffbd4f87ad7b3c4b67b91a8d593d40b6660da7"} Jul 24 13:51:19.892881 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:19.892579 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dtkts" event={"ID":"8ed75a6c-c13d-4524-b2cb-978fe5781631","Type":"ContainerDied","Data":"42858311d25a9db2086557d18a78c2fb3ffc9c2dee5b41fe510fafa301915bf0"} Jul 24 13:51:19.892881 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:19.892600 2557 scope.go:117] "RemoveContainer" containerID="4c90084817d579486808f889f6ffbd4f87ad7b3c4b67b91a8d593d40b6660da7" Jul 24 13:51:19.900966 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:19.900949 2557 scope.go:117] "RemoveContainer" containerID="4c90084817d579486808f889f6ffbd4f87ad7b3c4b67b91a8d593d40b6660da7" Jul 24 13:51:19.901277 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:51:19.901261 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c90084817d579486808f889f6ffbd4f87ad7b3c4b67b91a8d593d40b6660da7\": container with ID starting with 4c90084817d579486808f889f6ffbd4f87ad7b3c4b67b91a8d593d40b6660da7 not found: ID does not exist" containerID="4c90084817d579486808f889f6ffbd4f87ad7b3c4b67b91a8d593d40b6660da7" Jul 24 13:51:19.901322 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:19.901286 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c90084817d579486808f889f6ffbd4f87ad7b3c4b67b91a8d593d40b6660da7"} err="failed to get container status \"4c90084817d579486808f889f6ffbd4f87ad7b3c4b67b91a8d593d40b6660da7\": rpc error: code = NotFound desc = could not find container \"4c90084817d579486808f889f6ffbd4f87ad7b3c4b67b91a8d593d40b6660da7\": container with ID starting with 4c90084817d579486808f889f6ffbd4f87ad7b3c4b67b91a8d593d40b6660da7 not found: ID does not exist" Jul 24 13:51:19.911127 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:19.911102 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dtkts"] Jul 24 13:51:19.914425 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:19.914403 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dtkts"] Jul 24 13:51:19.968514 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:19.968475 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxflw\" (UniqueName: \"kubernetes.io/projected/8ed75a6c-c13d-4524-b2cb-978fe5781631-kube-api-access-kxflw\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:51:20.876382 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:20.876347 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ed75a6c-c13d-4524-b2cb-978fe5781631" path="/var/lib/kubelet/pods/8ed75a6c-c13d-4524-b2cb-978fe5781631/volumes" Jul 24 13:51:25.506934 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.506902 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68c8d45d56-n774n"] Jul 24 13:51:25.507363 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.507174 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ed75a6c-c13d-4524-b2cb-978fe5781631" containerName="authorino" Jul 24 13:51:25.507363 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.507184 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ed75a6c-c13d-4524-b2cb-978fe5781631" containerName="authorino" Jul 24 13:51:25.507363 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.507200 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bcee36d-c07d-4a31-a5f6-6f110c087030" containerName="authorino" Jul 24 13:51:25.507363 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.507206 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bcee36d-c07d-4a31-a5f6-6f110c087030" containerName="authorino" Jul 24 13:51:25.507363 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.507252 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ed75a6c-c13d-4524-b2cb-978fe5781631" containerName="authorino" Jul 24 13:51:25.507363 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.507261 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bcee36d-c07d-4a31-a5f6-6f110c087030" containerName="authorino" Jul 24 13:51:25.511698 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.511678 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c8d45d56-n774n" Jul 24 13:51:25.514420 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.514398 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 24 13:51:25.514544 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.514434 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 24 13:51:25.515214 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.515196 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vx2bh\"" Jul 24 13:51:25.524802 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.520897 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68c8d45d56-n774n"] Jul 24 13:51:25.608664 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.608623 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcpjx\" (UniqueName: \"kubernetes.io/projected/abddacb0-9ab4-4e86-b321-308d0c683d43-kube-api-access-wcpjx\") pod \"maas-controller-68c8d45d56-n774n\" (UID: \"abddacb0-9ab4-4e86-b321-308d0c683d43\") " pod="opendatahub/maas-controller-68c8d45d56-n774n" Jul 24 13:51:25.608884 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.608686 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abddacb0-9ab4-4e86-b321-308d0c683d43-webhook-cert\") pod \"maas-controller-68c8d45d56-n774n\" (UID: \"abddacb0-9ab4-4e86-b321-308d0c683d43\") " pod="opendatahub/maas-controller-68c8d45d56-n774n" Jul 24 13:51:25.691041 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.691003 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76f7f8bcdc-c64rv"] Jul 24 13:51:25.694542 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.694523 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" Jul 24 13:51:25.702422 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.702385 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f7f8bcdc-c64rv"] Jul 24 13:51:25.709389 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.709365 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abddacb0-9ab4-4e86-b321-308d0c683d43-webhook-cert\") pod \"maas-controller-68c8d45d56-n774n\" (UID: \"abddacb0-9ab4-4e86-b321-308d0c683d43\") " pod="opendatahub/maas-controller-68c8d45d56-n774n" Jul 24 13:51:25.709539 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.709434 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24024b7b-dcf1-41be-b3b8-58c52f2bbad3-webhook-cert\") pod \"maas-controller-76f7f8bcdc-c64rv\" (UID: \"24024b7b-dcf1-41be-b3b8-58c52f2bbad3\") " pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" Jul 24 13:51:25.709539 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.709463 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjnp6\" (UniqueName: \"kubernetes.io/projected/24024b7b-dcf1-41be-b3b8-58c52f2bbad3-kube-api-access-mjnp6\") pod \"maas-controller-76f7f8bcdc-c64rv\" (UID: \"24024b7b-dcf1-41be-b3b8-58c52f2bbad3\") " pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" Jul 24 13:51:25.709636 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:51:25.709537 2557 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 24 13:51:25.709636 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.709536 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcpjx\" (UniqueName: \"kubernetes.io/projected/abddacb0-9ab4-4e86-b321-308d0c683d43-kube-api-access-wcpjx\") pod \"maas-controller-68c8d45d56-n774n\" (UID: \"abddacb0-9ab4-4e86-b321-308d0c683d43\") " pod="opendatahub/maas-controller-68c8d45d56-n774n" Jul 24 13:51:25.709636 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:51:25.709624 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abddacb0-9ab4-4e86-b321-308d0c683d43-webhook-cert podName:abddacb0-9ab4-4e86-b321-308d0c683d43 nodeName:}" failed. No retries permitted until 2026-07-24 13:51:26.209604184 +0000 UTC m=+519.896365592 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/abddacb0-9ab4-4e86-b321-308d0c683d43-webhook-cert") pod "maas-controller-68c8d45d56-n774n" (UID: "abddacb0-9ab4-4e86-b321-308d0c683d43") : secret "maas-controller-webhook-cert" not found Jul 24 13:51:25.719643 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.719609 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcpjx\" (UniqueName: \"kubernetes.io/projected/abddacb0-9ab4-4e86-b321-308d0c683d43-kube-api-access-wcpjx\") pod \"maas-controller-68c8d45d56-n774n\" (UID: \"abddacb0-9ab4-4e86-b321-308d0c683d43\") " pod="opendatahub/maas-controller-68c8d45d56-n774n" Jul 24 13:51:25.810757 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.810657 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24024b7b-dcf1-41be-b3b8-58c52f2bbad3-webhook-cert\") pod \"maas-controller-76f7f8bcdc-c64rv\" (UID: \"24024b7b-dcf1-41be-b3b8-58c52f2bbad3\") " pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" Jul 24 13:51:25.810757 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.810700 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjnp6\" (UniqueName: \"kubernetes.io/projected/24024b7b-dcf1-41be-b3b8-58c52f2bbad3-kube-api-access-mjnp6\") pod \"maas-controller-76f7f8bcdc-c64rv\" (UID: \"24024b7b-dcf1-41be-b3b8-58c52f2bbad3\") " pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" Jul 24 13:51:25.813233 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.813210 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24024b7b-dcf1-41be-b3b8-58c52f2bbad3-webhook-cert\") pod \"maas-controller-76f7f8bcdc-c64rv\" (UID: \"24024b7b-dcf1-41be-b3b8-58c52f2bbad3\") " pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" Jul 24 13:51:25.818366 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:25.818340 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjnp6\" (UniqueName: \"kubernetes.io/projected/24024b7b-dcf1-41be-b3b8-58c52f2bbad3-kube-api-access-mjnp6\") pod \"maas-controller-76f7f8bcdc-c64rv\" (UID: \"24024b7b-dcf1-41be-b3b8-58c52f2bbad3\") " pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" Jul 24 13:51:26.005602 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:26.005564 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" Jul 24 13:51:26.138700 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:26.138670 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f7f8bcdc-c64rv"] Jul 24 13:51:26.142356 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:51:26.142331 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24024b7b_dcf1_41be_b3b8_58c52f2bbad3.slice/crio-851cd3f1b2cc57168458559407436f0f377e47dae9b4e493557320b9c5b1a897 WatchSource:0}: Error finding container 851cd3f1b2cc57168458559407436f0f377e47dae9b4e493557320b9c5b1a897: Status 404 returned error can't find the container with id 851cd3f1b2cc57168458559407436f0f377e47dae9b4e493557320b9c5b1a897 Jul 24 13:51:26.214200 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:26.214161 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abddacb0-9ab4-4e86-b321-308d0c683d43-webhook-cert\") pod \"maas-controller-68c8d45d56-n774n\" (UID: \"abddacb0-9ab4-4e86-b321-308d0c683d43\") " pod="opendatahub/maas-controller-68c8d45d56-n774n" Jul 24 13:51:26.216739 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:26.216718 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abddacb0-9ab4-4e86-b321-308d0c683d43-webhook-cert\") pod \"maas-controller-68c8d45d56-n774n\" (UID: \"abddacb0-9ab4-4e86-b321-308d0c683d43\") " pod="opendatahub/maas-controller-68c8d45d56-n774n" Jul 24 13:51:26.428167 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:26.428116 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c8d45d56-n774n" Jul 24 13:51:26.563895 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:26.563870 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68c8d45d56-n774n"] Jul 24 13:51:26.565791 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:51:26.565766 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabddacb0_9ab4_4e86_b321_308d0c683d43.slice/crio-432e2275fed11779ab628f59d5701308ef45ea57e417a4dbc8be355b81a04bd9 WatchSource:0}: Error finding container 432e2275fed11779ab628f59d5701308ef45ea57e417a4dbc8be355b81a04bd9: Status 404 returned error can't find the container with id 432e2275fed11779ab628f59d5701308ef45ea57e417a4dbc8be355b81a04bd9 Jul 24 13:51:26.924432 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:26.924390 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c8d45d56-n774n" event={"ID":"abddacb0-9ab4-4e86-b321-308d0c683d43","Type":"ContainerStarted","Data":"432e2275fed11779ab628f59d5701308ef45ea57e417a4dbc8be355b81a04bd9"} Jul 24 13:51:26.926194 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:26.926162 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" event={"ID":"24024b7b-dcf1-41be-b3b8-58c52f2bbad3","Type":"ContainerStarted","Data":"851cd3f1b2cc57168458559407436f0f377e47dae9b4e493557320b9c5b1a897"} Jul 24 13:51:28.936530 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:28.936485 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c8d45d56-n774n" event={"ID":"abddacb0-9ab4-4e86-b321-308d0c683d43","Type":"ContainerStarted","Data":"95ee3661207c4e0851546f4239c531d4821c737f6447d50016e34dfb68775477"} Jul 24 13:51:28.937007 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:28.936568 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68c8d45d56-n774n" Jul 24 13:51:28.937966 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:28.937946 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" event={"ID":"24024b7b-dcf1-41be-b3b8-58c52f2bbad3","Type":"ContainerStarted","Data":"4450538fa7a5b5c70706b7a989a4b2c1832d3b034574864b01f2fa3e094736b4"} Jul 24 13:51:28.938071 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:28.938061 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" Jul 24 13:51:28.981892 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:28.981841 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-68c8d45d56-n774n" podStartSLOduration=1.929564767 podStartE2EDuration="3.981823518s" totalImagesPullingTime="2.052258751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:51:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:51:28.976148533 +0000 UTC m=+522.662909950" watchObservedRunningTime="2026-07-24 13:51:28.981823518 +0000 UTC m=+522.668584926" Jul 24 13:51:29.026365 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:29.026306 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" podStartSLOduration=1.5605676929999999 podStartE2EDuration="4.026287882s" totalImagesPullingTime="2.465720189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:51:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:51:29.024277257 +0000 UTC m=+522.711038653" watchObservedRunningTime="2026-07-24 13:51:29.026287882 +0000 UTC m=+522.713049300" Jul 24 13:51:39.945380 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:39.945307 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" Jul 24 13:51:39.945850 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:39.945392 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68c8d45d56-n774n" Jul 24 13:51:40.009518 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:40.009479 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68c8d45d56-n774n"] Jul 24 13:51:40.009772 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:40.009697 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-68c8d45d56-n774n" podUID="abddacb0-9ab4-4e86-b321-308d0c683d43" containerName="manager" containerID="cri-o://95ee3661207c4e0851546f4239c531d4821c737f6447d50016e34dfb68775477" gracePeriod=30 Jul 24 13:51:40.252412 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:40.252389 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c8d45d56-n774n" Jul 24 13:51:40.323661 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:40.323623 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abddacb0-9ab4-4e86-b321-308d0c683d43-webhook-cert\") pod \"abddacb0-9ab4-4e86-b321-308d0c683d43\" (UID: \"abddacb0-9ab4-4e86-b321-308d0c683d43\") " Jul 24 13:51:40.323880 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:40.323672 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcpjx\" (UniqueName: \"kubernetes.io/projected/abddacb0-9ab4-4e86-b321-308d0c683d43-kube-api-access-wcpjx\") pod \"abddacb0-9ab4-4e86-b321-308d0c683d43\" (UID: \"abddacb0-9ab4-4e86-b321-308d0c683d43\") " Jul 24 13:51:40.326019 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:40.325986 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abddacb0-9ab4-4e86-b321-308d0c683d43-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "abddacb0-9ab4-4e86-b321-308d0c683d43" (UID: "abddacb0-9ab4-4e86-b321-308d0c683d43"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:51:40.326143 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:40.326059 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abddacb0-9ab4-4e86-b321-308d0c683d43-kube-api-access-wcpjx" (OuterVolumeSpecName: "kube-api-access-wcpjx") pod "abddacb0-9ab4-4e86-b321-308d0c683d43" (UID: "abddacb0-9ab4-4e86-b321-308d0c683d43"). InnerVolumeSpecName "kube-api-access-wcpjx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:51:40.424673 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:40.424632 2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abddacb0-9ab4-4e86-b321-308d0c683d43-webhook-cert\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:51:40.424673 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:40.424664 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcpjx\" (UniqueName: \"kubernetes.io/projected/abddacb0-9ab4-4e86-b321-308d0c683d43-kube-api-access-wcpjx\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:51:40.989108 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:40.989079 2557 generic.go:356] "Generic (PLEG): container finished" podID="abddacb0-9ab4-4e86-b321-308d0c683d43" containerID="95ee3661207c4e0851546f4239c531d4821c737f6447d50016e34dfb68775477" exitCode=0 Jul 24 13:51:40.989541 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:40.989129 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c8d45d56-n774n" Jul 24 13:51:40.989541 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:40.989132 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c8d45d56-n774n" event={"ID":"abddacb0-9ab4-4e86-b321-308d0c683d43","Type":"ContainerDied","Data":"95ee3661207c4e0851546f4239c531d4821c737f6447d50016e34dfb68775477"} Jul 24 13:51:40.989541 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:40.989176 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c8d45d56-n774n" event={"ID":"abddacb0-9ab4-4e86-b321-308d0c683d43","Type":"ContainerDied","Data":"432e2275fed11779ab628f59d5701308ef45ea57e417a4dbc8be355b81a04bd9"} Jul 24 13:51:40.989541 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:40.989198 2557 scope.go:117] "RemoveContainer" containerID="95ee3661207c4e0851546f4239c531d4821c737f6447d50016e34dfb68775477" Jul 24 13:51:40.997773 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:40.997755 2557 scope.go:117] "RemoveContainer" containerID="95ee3661207c4e0851546f4239c531d4821c737f6447d50016e34dfb68775477" Jul 24 13:51:40.998124 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:51:40.998107 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95ee3661207c4e0851546f4239c531d4821c737f6447d50016e34dfb68775477\": container with ID starting with 95ee3661207c4e0851546f4239c531d4821c737f6447d50016e34dfb68775477 not found: ID does not exist" containerID="95ee3661207c4e0851546f4239c531d4821c737f6447d50016e34dfb68775477" Jul 24 13:51:40.998179 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:40.998142 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95ee3661207c4e0851546f4239c531d4821c737f6447d50016e34dfb68775477"} err="failed to get container status \"95ee3661207c4e0851546f4239c531d4821c737f6447d50016e34dfb68775477\": rpc error: code = NotFound desc = could not find container \"95ee3661207c4e0851546f4239c531d4821c737f6447d50016e34dfb68775477\": container with ID starting with 95ee3661207c4e0851546f4239c531d4821c737f6447d50016e34dfb68775477 not found: ID does not exist" Jul 24 13:51:41.004621 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:41.004596 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68c8d45d56-n774n"] Jul 24 13:51:41.008714 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:41.008692 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-68c8d45d56-n774n"] Jul 24 13:51:42.876978 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:51:42.876939 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abddacb0-9ab4-4e86-b321-308d0c683d43" path="/var/lib/kubelet/pods/abddacb0-9ab4-4e86-b321-308d0c683d43/volumes" Jul 24 13:52:00.800430 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:00.800395 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-95d8b655f-2snl5"] Jul 24 13:52:00.800947 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:00.800833 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abddacb0-9ab4-4e86-b321-308d0c683d43" containerName="manager" Jul 24 13:52:00.800947 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:00.800848 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="abddacb0-9ab4-4e86-b321-308d0c683d43" containerName="manager" Jul 24 13:52:00.800947 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:00.800926 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="abddacb0-9ab4-4e86-b321-308d0c683d43" containerName="manager" Jul 24 13:52:00.808678 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:00.808656 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" Jul 24 13:52:00.810916 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:00.810888 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 24 13:52:00.811461 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:00.811315 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kzvrt\"" Jul 24 13:52:00.811461 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:00.811335 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 13:52:00.811627 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:00.811560 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 24 13:52:00.812579 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:00.812554 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-95d8b655f-2snl5"] Jul 24 13:52:00.902465 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:00.902436 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bd69357-578e-4bd2-8028-6369445e23ad-maas-api-tls\") pod \"maas-api-95d8b655f-2snl5\" (UID: \"5bd69357-578e-4bd2-8028-6369445e23ad\") " pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" Jul 24 13:52:00.902465 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:00.902470 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bd69357-578e-4bd2-8028-6369445e23ad-openshift-service-ca\") pod \"maas-api-95d8b655f-2snl5\" (UID: \"5bd69357-578e-4bd2-8028-6369445e23ad\") " pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" Jul 24 13:52:00.902736 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:00.902497 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x95pv\" (UniqueName: \"kubernetes.io/projected/5bd69357-578e-4bd2-8028-6369445e23ad-kube-api-access-x95pv\") pod \"maas-api-95d8b655f-2snl5\" (UID: \"5bd69357-578e-4bd2-8028-6369445e23ad\") " pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" Jul 24 13:52:01.003786 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:01.003748 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bd69357-578e-4bd2-8028-6369445e23ad-maas-api-tls\") pod \"maas-api-95d8b655f-2snl5\" (UID: \"5bd69357-578e-4bd2-8028-6369445e23ad\") " pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" Jul 24 13:52:01.003786 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:01.003786 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bd69357-578e-4bd2-8028-6369445e23ad-openshift-service-ca\") pod \"maas-api-95d8b655f-2snl5\" (UID: \"5bd69357-578e-4bd2-8028-6369445e23ad\") " pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" Jul 24 13:52:01.004080 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:01.003837 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x95pv\" (UniqueName: \"kubernetes.io/projected/5bd69357-578e-4bd2-8028-6369445e23ad-kube-api-access-x95pv\") pod \"maas-api-95d8b655f-2snl5\" (UID: \"5bd69357-578e-4bd2-8028-6369445e23ad\") " pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" Jul 24 13:52:01.004080 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:52:01.003925 2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 24 13:52:01.004080 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:52:01.003994 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bd69357-578e-4bd2-8028-6369445e23ad-maas-api-tls podName:5bd69357-578e-4bd2-8028-6369445e23ad nodeName:}" failed. No retries permitted until 2026-07-24 13:52:01.503976717 +0000 UTC m=+555.190738111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5bd69357-578e-4bd2-8028-6369445e23ad-maas-api-tls") pod "maas-api-95d8b655f-2snl5" (UID: "5bd69357-578e-4bd2-8028-6369445e23ad") : secret "maas-api-serving-cert" not found Jul 24 13:52:01.004703 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:01.004681 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bd69357-578e-4bd2-8028-6369445e23ad-openshift-service-ca\") pod \"maas-api-95d8b655f-2snl5\" (UID: \"5bd69357-578e-4bd2-8028-6369445e23ad\") " pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" Jul 24 13:52:01.012405 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:01.012374 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x95pv\" (UniqueName: \"kubernetes.io/projected/5bd69357-578e-4bd2-8028-6369445e23ad-kube-api-access-x95pv\") pod \"maas-api-95d8b655f-2snl5\" (UID: \"5bd69357-578e-4bd2-8028-6369445e23ad\") " pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" Jul 24 13:52:01.507850 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:01.507796 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bd69357-578e-4bd2-8028-6369445e23ad-maas-api-tls\") pod \"maas-api-95d8b655f-2snl5\" (UID: \"5bd69357-578e-4bd2-8028-6369445e23ad\") " pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" Jul 24 13:52:01.510360 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:01.510329 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bd69357-578e-4bd2-8028-6369445e23ad-maas-api-tls\") pod \"maas-api-95d8b655f-2snl5\" (UID: \"5bd69357-578e-4bd2-8028-6369445e23ad\") " pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" Jul 24 13:52:01.720952 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:01.720918 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" Jul 24 13:52:01.861050 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:01.861024 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-95d8b655f-2snl5"] Jul 24 13:52:01.863781 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:52:01.863749 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd69357_578e_4bd2_8028_6369445e23ad.slice/crio-ec10705933342f86dbf46346be9659f403076083625ce8d6f9d934941c15c506 WatchSource:0}: Error finding container ec10705933342f86dbf46346be9659f403076083625ce8d6f9d934941c15c506: Status 404 returned error can't find the container with id ec10705933342f86dbf46346be9659f403076083625ce8d6f9d934941c15c506 Jul 24 13:52:02.076871 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:02.076749 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" event={"ID":"5bd69357-578e-4bd2-8028-6369445e23ad","Type":"ContainerStarted","Data":"ec10705933342f86dbf46346be9659f403076083625ce8d6f9d934941c15c506"} Jul 24 13:52:04.087385 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:04.087353 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" event={"ID":"5bd69357-578e-4bd2-8028-6369445e23ad","Type":"ContainerStarted","Data":"ba979fc119b358df9cb1b46ea2ad5d4df1ca4fa214741b6f8fc16f4c66523374"} Jul 24 13:52:04.087793 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:04.087467 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" Jul 24 13:52:04.125161 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:04.125101 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" podStartSLOduration=2.690626194 podStartE2EDuration="4.125081317s" totalImagesPullingTime="1.434455123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:52:04.123835586 +0000 UTC m=+557.810597002" watchObservedRunningTime="2026-07-24 13:52:04.125081317 +0000 UTC m=+557.811842745" Jul 24 13:52:10.097577 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:10.097543 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-95d8b655f-2snl5" Jul 24 13:52:36.481834 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.481772 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm"] Jul 24 13:52:36.486999 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.486981 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.489042 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.489019 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 24 13:52:36.489733 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.489709 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hrqmm\"" Jul 24 13:52:36.489864 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.489766 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 24 13:52:36.489864 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.489775 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 24 13:52:36.495665 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.495644 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm"] Jul 24 13:52:36.606406 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.606369 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3060d59-d4a8-4638-a7bb-824c9b8ca918-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.606575 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.606437 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3060d59-d4a8-4638-a7bb-824c9b8ca918-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.606575 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.606459 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3060d59-d4a8-4638-a7bb-824c9b8ca918-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.606575 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.606479 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3060d59-d4a8-4638-a7bb-824c9b8ca918-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.606575 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.606498 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3060d59-d4a8-4638-a7bb-824c9b8ca918-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.606703 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.606618 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w22n4\" (UniqueName: \"kubernetes.io/projected/f3060d59-d4a8-4638-a7bb-824c9b8ca918-kube-api-access-w22n4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.707374 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.707334 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w22n4\" (UniqueName: \"kubernetes.io/projected/f3060d59-d4a8-4638-a7bb-824c9b8ca918-kube-api-access-w22n4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.707374 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.707379 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3060d59-d4a8-4638-a7bb-824c9b8ca918-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.707611 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.707406 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3060d59-d4a8-4638-a7bb-824c9b8ca918-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.707611 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.707424 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3060d59-d4a8-4638-a7bb-824c9b8ca918-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.707611 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.707444 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3060d59-d4a8-4638-a7bb-824c9b8ca918-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.707611 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.707460 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3060d59-d4a8-4638-a7bb-824c9b8ca918-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.707851 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.707801 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3060d59-d4a8-4638-a7bb-824c9b8ca918-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.707933 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.707904 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3060d59-d4a8-4638-a7bb-824c9b8ca918-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.707992 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.707965 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3060d59-d4a8-4638-a7bb-824c9b8ca918-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.709981 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.709959 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3060d59-d4a8-4638-a7bb-824c9b8ca918-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.710174 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.710154 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3060d59-d4a8-4638-a7bb-824c9b8ca918-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.715209 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.715187 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w22n4\" (UniqueName: \"kubernetes.io/projected/f3060d59-d4a8-4638-a7bb-824c9b8ca918-kube-api-access-w22n4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-lcgdm\" (UID: \"f3060d59-d4a8-4638-a7bb-824c9b8ca918\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.797349 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.797257 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:36.931992 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:36.931969 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm"] Jul 24 13:52:36.934371 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:52:36.934345 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3060d59_d4a8_4638_a7bb_824c9b8ca918.slice/crio-8ab49766e1f7cedb4b682548361b92ec5f83fea5850b45097f913e104cc3c06c WatchSource:0}: Error finding container 8ab49766e1f7cedb4b682548361b92ec5f83fea5850b45097f913e104cc3c06c: Status 404 returned error can't find the container with id 8ab49766e1f7cedb4b682548361b92ec5f83fea5850b45097f913e104cc3c06c Jul 24 13:52:37.226297 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:37.226253 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" event={"ID":"f3060d59-d4a8-4638-a7bb-824c9b8ca918","Type":"ContainerStarted","Data":"8ab49766e1f7cedb4b682548361b92ec5f83fea5850b45097f913e104cc3c06c"} Jul 24 13:52:38.232058 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:38.231969 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" event={"ID":"f3060d59-d4a8-4638-a7bb-824c9b8ca918","Type":"ContainerStarted","Data":"597d3e2a5ecf649eebed51d27d483b09854107d46067fb4ea34a7031427a06a6"} Jul 24 13:52:38.249323 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:38.249279 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" podStartSLOduration=1.252164676 podStartE2EDuration="2.249261379s" totalImagesPullingTime="997.096703ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:52:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:52:38.248021162 +0000 UTC m=+591.934782579" watchObservedRunningTime="2026-07-24 13:52:38.249261379 +0000 UTC m=+591.936022796" Jul 24 13:52:46.797901 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:46.797862 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:46.798391 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:46.797952 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:46.810675 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:46.810648 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:52:47.281137 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:52:47.281109 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-lcgdm" Jul 24 13:53:06.189836 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.189789 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj"] Jul 24 13:53:06.192936 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.192917 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.197442 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.197423 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 24 13:53:06.204638 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.204614 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj"] Jul 24 13:53:06.262446 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.262398 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4cac94b-46e7-4ac1-a3c5-8025350837d8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.262446 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.262457 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4cac94b-46e7-4ac1-a3c5-8025350837d8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.262672 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.262488 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4cac94b-46e7-4ac1-a3c5-8025350837d8-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.262672 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.262504 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4cac94b-46e7-4ac1-a3c5-8025350837d8-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.262672 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.262527 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4cac94b-46e7-4ac1-a3c5-8025350837d8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.262672 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.262588 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rshn\" (UniqueName: \"kubernetes.io/projected/c4cac94b-46e7-4ac1-a3c5-8025350837d8-kube-api-access-6rshn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.363533 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.363494 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rshn\" (UniqueName: \"kubernetes.io/projected/c4cac94b-46e7-4ac1-a3c5-8025350837d8-kube-api-access-6rshn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.363732 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.363571 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4cac94b-46e7-4ac1-a3c5-8025350837d8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.363732 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.363607 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4cac94b-46e7-4ac1-a3c5-8025350837d8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.363732 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.363640 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4cac94b-46e7-4ac1-a3c5-8025350837d8-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.363732 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.363664 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4cac94b-46e7-4ac1-a3c5-8025350837d8-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.363732 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.363699 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4cac94b-46e7-4ac1-a3c5-8025350837d8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.364078 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.364058 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4cac94b-46e7-4ac1-a3c5-8025350837d8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.364161 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.364137 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4cac94b-46e7-4ac1-a3c5-8025350837d8-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.364311 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.364293 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4cac94b-46e7-4ac1-a3c5-8025350837d8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.366252 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.366230 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4cac94b-46e7-4ac1-a3c5-8025350837d8-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.366852 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.366834 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4cac94b-46e7-4ac1-a3c5-8025350837d8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.370592 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.370570 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rshn\" (UniqueName: \"kubernetes.io/projected/c4cac94b-46e7-4ac1-a3c5-8025350837d8-kube-api-access-6rshn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-x26sj\" (UID: \"c4cac94b-46e7-4ac1-a3c5-8025350837d8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.502758 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.502669 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:06.633606 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.633581 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj"] Jul 24 13:53:06.635627 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:53:06.635600 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4cac94b_46e7_4ac1_a3c5_8025350837d8.slice/crio-acb18f0b7c3695a30bc2a806c79e4b66c10aad99f14f868f3207378e335b28d6 WatchSource:0}: Error finding container acb18f0b7c3695a30bc2a806c79e4b66c10aad99f14f868f3207378e335b28d6: Status 404 returned error can't find the container with id acb18f0b7c3695a30bc2a806c79e4b66c10aad99f14f868f3207378e335b28d6 Jul 24 13:53:06.636906 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:06.636887 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:53:07.349417 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:07.349387 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" event={"ID":"c4cac94b-46e7-4ac1-a3c5-8025350837d8","Type":"ContainerStarted","Data":"ba48579f3ef8d6641a75cf5d0be54784503937eadf72c01696c091f29593d3c7"} Jul 24 13:53:07.349847 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:07.349434 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" event={"ID":"c4cac94b-46e7-4ac1-a3c5-8025350837d8","Type":"ContainerStarted","Data":"acb18f0b7c3695a30bc2a806c79e4b66c10aad99f14f868f3207378e335b28d6"} Jul 24 13:53:07.368710 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:07.368655 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" podStartSLOduration=1.169639833 podStartE2EDuration="1.368639403s" totalImagesPullingTime="198.99957ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:53:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:53:07.36708989 +0000 UTC m=+621.053851307" watchObservedRunningTime="2026-07-24 13:53:07.368639403 +0000 UTC m=+621.055400820" Jul 24 13:53:16.503159 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:16.503119 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:16.503560 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:16.503221 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:16.515389 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:16.515361 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:17.402949 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:17.402909 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-x26sj" Jul 24 13:53:18.389398 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.389365 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q"] Jul 24 13:53:18.392464 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.392440 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.394923 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.394903 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 24 13:53:18.403736 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.403711 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q"] Jul 24 13:53:18.468196 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.468161 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c48264f-fd02-416b-a63f-98944fce2771-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.468453 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.468209 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c48264f-fd02-416b-a63f-98944fce2771-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.468453 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.468309 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c48264f-fd02-416b-a63f-98944fce2771-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.468453 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.468381 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c48264f-fd02-416b-a63f-98944fce2771-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.468602 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.468465 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l9rs\" (UniqueName: \"kubernetes.io/projected/7c48264f-fd02-416b-a63f-98944fce2771-kube-api-access-5l9rs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.468602 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.468549 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c48264f-fd02-416b-a63f-98944fce2771-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.569212 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.569182 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l9rs\" (UniqueName: \"kubernetes.io/projected/7c48264f-fd02-416b-a63f-98944fce2771-kube-api-access-5l9rs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.569438 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.569230 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c48264f-fd02-416b-a63f-98944fce2771-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.569438 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.569389 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c48264f-fd02-416b-a63f-98944fce2771-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.569438 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.569433 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c48264f-fd02-416b-a63f-98944fce2771-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.569611 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.569489 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c48264f-fd02-416b-a63f-98944fce2771-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.569611 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.569544 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c48264f-fd02-416b-a63f-98944fce2771-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.569934 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.569910 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c48264f-fd02-416b-a63f-98944fce2771-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.570242 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.570217 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c48264f-fd02-416b-a63f-98944fce2771-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.570395 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.570290 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c48264f-fd02-416b-a63f-98944fce2771-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.572454 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.572431 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c48264f-fd02-416b-a63f-98944fce2771-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.572564 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.572510 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c48264f-fd02-416b-a63f-98944fce2771-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.577877 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.577854 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l9rs\" (UniqueName: \"kubernetes.io/projected/7c48264f-fd02-416b-a63f-98944fce2771-kube-api-access-5l9rs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q\" (UID: \"7c48264f-fd02-416b-a63f-98944fce2771\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.704539 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.704453 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:18.838110 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:18.838088 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q"] Jul 24 13:53:18.840996 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:53:18.840968 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c48264f_fd02_416b_a63f_98944fce2771.slice/crio-c8dd73a0b89d632863549ac42297f3583c321e2c499710e5664680c29b4143a4 WatchSource:0}: Error finding container c8dd73a0b89d632863549ac42297f3583c321e2c499710e5664680c29b4143a4: Status 404 returned error can't find the container with id c8dd73a0b89d632863549ac42297f3583c321e2c499710e5664680c29b4143a4 Jul 24 13:53:19.402164 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:19.402127 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" event={"ID":"7c48264f-fd02-416b-a63f-98944fce2771","Type":"ContainerStarted","Data":"b1ee724c86602b1871ae5bb6ed7eb2794ab25a0e05146b433c40f92fe4ee5464"} Jul 24 13:53:19.402164 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:19.402165 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" event={"ID":"7c48264f-fd02-416b-a63f-98944fce2771","Type":"ContainerStarted","Data":"c8dd73a0b89d632863549ac42297f3583c321e2c499710e5664680c29b4143a4"} Jul 24 13:53:19.420222 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:19.420163 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" podStartSLOduration=1.243502303 podStartE2EDuration="1.420146121s" totalImagesPullingTime="176.643818ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:53:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:53:19.418669497 +0000 UTC m=+633.105430918" watchObservedRunningTime="2026-07-24 13:53:19.420146121 +0000 UTC m=+633.106907537" Jul 24 13:53:28.705615 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:28.705573 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:28.706072 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:28.705630 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:28.718183 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:28.718157 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:53:29.457709 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:53:29.457680 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q" Jul 24 13:54:54.215095 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.215013 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76f7f8bcdc-c64rv"] Jul 24 13:54:54.215532 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.215215 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" podUID="24024b7b-dcf1-41be-b3b8-58c52f2bbad3" containerName="manager" containerID="cri-o://4450538fa7a5b5c70706b7a989a4b2c1832d3b034574864b01f2fa3e094736b4" gracePeriod=30 Jul 24 13:54:54.458981 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.458957 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" Jul 24 13:54:54.537628 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.537549 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24024b7b-dcf1-41be-b3b8-58c52f2bbad3-webhook-cert\") pod \"24024b7b-dcf1-41be-b3b8-58c52f2bbad3\" (UID: \"24024b7b-dcf1-41be-b3b8-58c52f2bbad3\") " Jul 24 13:54:54.537776 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.537644 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjnp6\" (UniqueName: \"kubernetes.io/projected/24024b7b-dcf1-41be-b3b8-58c52f2bbad3-kube-api-access-mjnp6\") pod \"24024b7b-dcf1-41be-b3b8-58c52f2bbad3\" (UID: \"24024b7b-dcf1-41be-b3b8-58c52f2bbad3\") " Jul 24 13:54:54.539749 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.539722 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24024b7b-dcf1-41be-b3b8-58c52f2bbad3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "24024b7b-dcf1-41be-b3b8-58c52f2bbad3" (UID: "24024b7b-dcf1-41be-b3b8-58c52f2bbad3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:54:54.539874 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.539769 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24024b7b-dcf1-41be-b3b8-58c52f2bbad3-kube-api-access-mjnp6" (OuterVolumeSpecName: "kube-api-access-mjnp6") pod "24024b7b-dcf1-41be-b3b8-58c52f2bbad3" (UID: "24024b7b-dcf1-41be-b3b8-58c52f2bbad3"). InnerVolumeSpecName "kube-api-access-mjnp6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:54:54.638694 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.638640 2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24024b7b-dcf1-41be-b3b8-58c52f2bbad3-webhook-cert\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:54:54.638694 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.638685 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjnp6\" (UniqueName: \"kubernetes.io/projected/24024b7b-dcf1-41be-b3b8-58c52f2bbad3-kube-api-access-mjnp6\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:54:54.798261 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.798180 2557 generic.go:356] "Generic (PLEG): container finished" podID="24024b7b-dcf1-41be-b3b8-58c52f2bbad3" containerID="4450538fa7a5b5c70706b7a989a4b2c1832d3b034574864b01f2fa3e094736b4" exitCode=0 Jul 24 13:54:54.798261 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.798228 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" Jul 24 13:54:54.798440 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.798271 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" event={"ID":"24024b7b-dcf1-41be-b3b8-58c52f2bbad3","Type":"ContainerDied","Data":"4450538fa7a5b5c70706b7a989a4b2c1832d3b034574864b01f2fa3e094736b4"} Jul 24 13:54:54.798440 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.798328 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f7f8bcdc-c64rv" event={"ID":"24024b7b-dcf1-41be-b3b8-58c52f2bbad3","Type":"ContainerDied","Data":"851cd3f1b2cc57168458559407436f0f377e47dae9b4e493557320b9c5b1a897"} Jul 24 13:54:54.798440 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.798357 2557 scope.go:117] "RemoveContainer" containerID="4450538fa7a5b5c70706b7a989a4b2c1832d3b034574864b01f2fa3e094736b4" Jul 24 13:54:54.807198 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.807180 2557 scope.go:117] "RemoveContainer" containerID="4450538fa7a5b5c70706b7a989a4b2c1832d3b034574864b01f2fa3e094736b4" Jul 24 13:54:54.807474 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:54:54.807455 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4450538fa7a5b5c70706b7a989a4b2c1832d3b034574864b01f2fa3e094736b4\": container with ID starting with 4450538fa7a5b5c70706b7a989a4b2c1832d3b034574864b01f2fa3e094736b4 not found: ID does not exist" containerID="4450538fa7a5b5c70706b7a989a4b2c1832d3b034574864b01f2fa3e094736b4" Jul 24 13:54:54.807520 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.807482 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4450538fa7a5b5c70706b7a989a4b2c1832d3b034574864b01f2fa3e094736b4"} err="failed to get container status \"4450538fa7a5b5c70706b7a989a4b2c1832d3b034574864b01f2fa3e094736b4\": rpc error: code = NotFound desc = could not find container \"4450538fa7a5b5c70706b7a989a4b2c1832d3b034574864b01f2fa3e094736b4\": container with ID starting with 4450538fa7a5b5c70706b7a989a4b2c1832d3b034574864b01f2fa3e094736b4 not found: ID does not exist" Jul 24 13:54:54.818100 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.818067 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76f7f8bcdc-c64rv"] Jul 24 13:54:54.826241 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.822725 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-76f7f8bcdc-c64rv"] Jul 24 13:54:54.877639 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:54:54.877603 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24024b7b-dcf1-41be-b3b8-58c52f2bbad3" path="/var/lib/kubelet/pods/24024b7b-dcf1-41be-b3b8-58c52f2bbad3/volumes" Jul 24 13:55:55.677297 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:55.677244 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76f7f8bcdc-x5q7t"] Jul 24 13:55:55.677717 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:55.677581 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24024b7b-dcf1-41be-b3b8-58c52f2bbad3" containerName="manager" Jul 24 13:55:55.677717 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:55.677593 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="24024b7b-dcf1-41be-b3b8-58c52f2bbad3" containerName="manager" Jul 24 13:55:55.677717 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:55.677645 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="24024b7b-dcf1-41be-b3b8-58c52f2bbad3" containerName="manager" Jul 24 13:55:55.681518 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:55.681501 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" Jul 24 13:55:55.684591 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:55.684564 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 24 13:55:55.684718 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:55.684566 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 24 13:55:55.684718 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:55.684564 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vx2bh\"" Jul 24 13:55:55.689765 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:55.689740 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f7f8bcdc-x5q7t"] Jul 24 13:55:55.772069 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:55.772030 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa6e6fd8-f625-46e2-be3a-880646f7781a-webhook-cert\") pod \"maas-controller-76f7f8bcdc-x5q7t\" (UID: \"aa6e6fd8-f625-46e2-be3a-880646f7781a\") " pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" Jul 24 13:55:55.772069 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:55.772075 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzw9x\" (UniqueName: \"kubernetes.io/projected/aa6e6fd8-f625-46e2-be3a-880646f7781a-kube-api-access-mzw9x\") pod \"maas-controller-76f7f8bcdc-x5q7t\" (UID: \"aa6e6fd8-f625-46e2-be3a-880646f7781a\") " pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" Jul 24 13:55:55.873117 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:55.873084 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa6e6fd8-f625-46e2-be3a-880646f7781a-webhook-cert\") pod \"maas-controller-76f7f8bcdc-x5q7t\" (UID: \"aa6e6fd8-f625-46e2-be3a-880646f7781a\") " pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" Jul 24 13:55:55.873117 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:55.873124 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzw9x\" (UniqueName: \"kubernetes.io/projected/aa6e6fd8-f625-46e2-be3a-880646f7781a-kube-api-access-mzw9x\") pod \"maas-controller-76f7f8bcdc-x5q7t\" (UID: \"aa6e6fd8-f625-46e2-be3a-880646f7781a\") " pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" Jul 24 13:55:55.875694 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:55.875674 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa6e6fd8-f625-46e2-be3a-880646f7781a-webhook-cert\") pod \"maas-controller-76f7f8bcdc-x5q7t\" (UID: \"aa6e6fd8-f625-46e2-be3a-880646f7781a\") " pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" Jul 24 13:55:55.882137 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:55.882111 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzw9x\" (UniqueName: \"kubernetes.io/projected/aa6e6fd8-f625-46e2-be3a-880646f7781a-kube-api-access-mzw9x\") pod \"maas-controller-76f7f8bcdc-x5q7t\" (UID: \"aa6e6fd8-f625-46e2-be3a-880646f7781a\") " pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" Jul 24 13:55:55.992778 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:55.992671 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" Jul 24 13:55:56.131789 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:56.131755 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f7f8bcdc-x5q7t"] Jul 24 13:55:56.133943 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:55:56.133914 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa6e6fd8_f625_46e2_be3a_880646f7781a.slice/crio-a9e90ba1857aa21ae1d16cef4b1967a4520b188c7b513c01b73f3dda9eb4458e WatchSource:0}: Error finding container a9e90ba1857aa21ae1d16cef4b1967a4520b188c7b513c01b73f3dda9eb4458e: Status 404 returned error can't find the container with id a9e90ba1857aa21ae1d16cef4b1967a4520b188c7b513c01b73f3dda9eb4458e Jul 24 13:55:57.061380 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:57.061294 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" event={"ID":"aa6e6fd8-f625-46e2-be3a-880646f7781a","Type":"ContainerStarted","Data":"56fd7277a11a95c785b23a31046450368fabe0c6a6e2d9bfbc6e2fdfe3485fa4"} Jul 24 13:55:57.061380 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:57.061343 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" event={"ID":"aa6e6fd8-f625-46e2-be3a-880646f7781a","Type":"ContainerStarted","Data":"a9e90ba1857aa21ae1d16cef4b1967a4520b188c7b513c01b73f3dda9eb4458e"} Jul 24 13:55:57.061380 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:57.061381 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" Jul 24 13:55:57.077705 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:55:57.077654 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" podStartSLOduration=1.604896347 podStartE2EDuration="2.077637412s" totalImagesPullingTime="472.741065ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:55:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:55:57.076707924 +0000 UTC m=+790.763469341" watchObservedRunningTime="2026-07-24 13:55:57.077637412 +0000 UTC m=+790.764398828" Jul 24 13:56:08.067963 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:08.067884 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" Jul 24 13:56:12.701194 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:12.701157 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76f7f8bcdc-x5q7t"] Jul 24 13:56:12.701602 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:12.701363 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" podUID="aa6e6fd8-f625-46e2-be3a-880646f7781a" containerName="manager" containerID="cri-o://56fd7277a11a95c785b23a31046450368fabe0c6a6e2d9bfbc6e2fdfe3485fa4" gracePeriod=30 Jul 24 13:56:12.940649 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:12.940626 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" Jul 24 13:56:13.114180 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:13.114151 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa6e6fd8-f625-46e2-be3a-880646f7781a-webhook-cert\") pod \"aa6e6fd8-f625-46e2-be3a-880646f7781a\" (UID: \"aa6e6fd8-f625-46e2-be3a-880646f7781a\") " Jul 24 13:56:13.114468 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:13.114207 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzw9x\" (UniqueName: \"kubernetes.io/projected/aa6e6fd8-f625-46e2-be3a-880646f7781a-kube-api-access-mzw9x\") pod \"aa6e6fd8-f625-46e2-be3a-880646f7781a\" (UID: \"aa6e6fd8-f625-46e2-be3a-880646f7781a\") " Jul 24 13:56:13.124213 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:13.124182 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa6e6fd8-f625-46e2-be3a-880646f7781a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "aa6e6fd8-f625-46e2-be3a-880646f7781a" (UID: "aa6e6fd8-f625-46e2-be3a-880646f7781a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:56:13.124345 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:13.124213 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa6e6fd8-f625-46e2-be3a-880646f7781a-kube-api-access-mzw9x" (OuterVolumeSpecName: "kube-api-access-mzw9x") pod "aa6e6fd8-f625-46e2-be3a-880646f7781a" (UID: "aa6e6fd8-f625-46e2-be3a-880646f7781a"). InnerVolumeSpecName "kube-api-access-mzw9x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:56:13.136587 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:13.136555 2557 generic.go:356] "Generic (PLEG): container finished" podID="aa6e6fd8-f625-46e2-be3a-880646f7781a" containerID="56fd7277a11a95c785b23a31046450368fabe0c6a6e2d9bfbc6e2fdfe3485fa4" exitCode=0 Jul 24 13:56:13.136740 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:13.136673 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" Jul 24 13:56:13.136740 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:13.136678 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" event={"ID":"aa6e6fd8-f625-46e2-be3a-880646f7781a","Type":"ContainerDied","Data":"56fd7277a11a95c785b23a31046450368fabe0c6a6e2d9bfbc6e2fdfe3485fa4"} Jul 24 13:56:13.136896 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:13.136766 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f7f8bcdc-x5q7t" event={"ID":"aa6e6fd8-f625-46e2-be3a-880646f7781a","Type":"ContainerDied","Data":"a9e90ba1857aa21ae1d16cef4b1967a4520b188c7b513c01b73f3dda9eb4458e"} Jul 24 13:56:13.136896 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:13.136801 2557 scope.go:117] "RemoveContainer" containerID="56fd7277a11a95c785b23a31046450368fabe0c6a6e2d9bfbc6e2fdfe3485fa4" Jul 24 13:56:13.145716 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:13.145697 2557 scope.go:117] "RemoveContainer" containerID="56fd7277a11a95c785b23a31046450368fabe0c6a6e2d9bfbc6e2fdfe3485fa4" Jul 24 13:56:13.146031 ip-10-0-128-250 kubenswrapper[2557]: E0724 13:56:13.146007 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56fd7277a11a95c785b23a31046450368fabe0c6a6e2d9bfbc6e2fdfe3485fa4\": container with ID starting with 56fd7277a11a95c785b23a31046450368fabe0c6a6e2d9bfbc6e2fdfe3485fa4 not found: ID does not exist" containerID="56fd7277a11a95c785b23a31046450368fabe0c6a6e2d9bfbc6e2fdfe3485fa4" Jul 24 13:56:13.146085 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:13.146044 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56fd7277a11a95c785b23a31046450368fabe0c6a6e2d9bfbc6e2fdfe3485fa4"} err="failed to get container status \"56fd7277a11a95c785b23a31046450368fabe0c6a6e2d9bfbc6e2fdfe3485fa4\": rpc error: code = NotFound desc = could not find container \"56fd7277a11a95c785b23a31046450368fabe0c6a6e2d9bfbc6e2fdfe3485fa4\": container with ID starting with 56fd7277a11a95c785b23a31046450368fabe0c6a6e2d9bfbc6e2fdfe3485fa4 not found: ID does not exist" Jul 24 13:56:13.157929 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:13.157900 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76f7f8bcdc-x5q7t"] Jul 24 13:56:13.162108 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:13.162082 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-76f7f8bcdc-x5q7t"] Jul 24 13:56:13.215022 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:13.214993 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzw9x\" (UniqueName: \"kubernetes.io/projected/aa6e6fd8-f625-46e2-be3a-880646f7781a-kube-api-access-mzw9x\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:56:13.215022 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:13.215017 2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa6e6fd8-f625-46e2-be3a-880646f7781a-webhook-cert\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 13:56:14.002900 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.002866 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76f7f8bcdc-wcq4d"] Jul 24 13:56:14.003288 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.003157 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa6e6fd8-f625-46e2-be3a-880646f7781a" containerName="manager" Jul 24 13:56:14.003288 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.003167 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa6e6fd8-f625-46e2-be3a-880646f7781a" containerName="manager" Jul 24 13:56:14.003288 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.003232 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa6e6fd8-f625-46e2-be3a-880646f7781a" containerName="manager" Jul 24 13:56:14.007397 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.007381 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f7f8bcdc-wcq4d" Jul 24 13:56:14.009750 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.009730 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 24 13:56:14.010601 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.010583 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 24 13:56:14.010673 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.010602 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vx2bh\"" Jul 24 13:56:14.015838 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.015798 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f7f8bcdc-wcq4d"] Jul 24 13:56:14.122507 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.122465 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cf7b06e-53ff-49ba-b7b8-4fbfe3c87fa9-webhook-cert\") pod \"maas-controller-76f7f8bcdc-wcq4d\" (UID: \"6cf7b06e-53ff-49ba-b7b8-4fbfe3c87fa9\") " pod="opendatahub/maas-controller-76f7f8bcdc-wcq4d" Jul 24 13:56:14.122673 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.122518 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ktbk\" (UniqueName: \"kubernetes.io/projected/6cf7b06e-53ff-49ba-b7b8-4fbfe3c87fa9-kube-api-access-6ktbk\") pod \"maas-controller-76f7f8bcdc-wcq4d\" (UID: \"6cf7b06e-53ff-49ba-b7b8-4fbfe3c87fa9\") " pod="opendatahub/maas-controller-76f7f8bcdc-wcq4d" Jul 24 13:56:14.223742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.223707 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cf7b06e-53ff-49ba-b7b8-4fbfe3c87fa9-webhook-cert\") pod \"maas-controller-76f7f8bcdc-wcq4d\" (UID: \"6cf7b06e-53ff-49ba-b7b8-4fbfe3c87fa9\") " pod="opendatahub/maas-controller-76f7f8bcdc-wcq4d" Jul 24 13:56:14.223742 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.223759 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ktbk\" (UniqueName: \"kubernetes.io/projected/6cf7b06e-53ff-49ba-b7b8-4fbfe3c87fa9-kube-api-access-6ktbk\") pod \"maas-controller-76f7f8bcdc-wcq4d\" (UID: \"6cf7b06e-53ff-49ba-b7b8-4fbfe3c87fa9\") " pod="opendatahub/maas-controller-76f7f8bcdc-wcq4d" Jul 24 13:56:14.226385 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.226356 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6cf7b06e-53ff-49ba-b7b8-4fbfe3c87fa9-webhook-cert\") pod \"maas-controller-76f7f8bcdc-wcq4d\" (UID: \"6cf7b06e-53ff-49ba-b7b8-4fbfe3c87fa9\") " pod="opendatahub/maas-controller-76f7f8bcdc-wcq4d" Jul 24 13:56:14.233337 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.233315 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ktbk\" (UniqueName: \"kubernetes.io/projected/6cf7b06e-53ff-49ba-b7b8-4fbfe3c87fa9-kube-api-access-6ktbk\") pod \"maas-controller-76f7f8bcdc-wcq4d\" (UID: \"6cf7b06e-53ff-49ba-b7b8-4fbfe3c87fa9\") " pod="opendatahub/maas-controller-76f7f8bcdc-wcq4d" Jul 24 13:56:14.319034 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.318928 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f7f8bcdc-wcq4d" Jul 24 13:56:14.450186 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.450106 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f7f8bcdc-wcq4d"] Jul 24 13:56:14.452476 ip-10-0-128-250 kubenswrapper[2557]: W0724 13:56:14.452447 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cf7b06e_53ff_49ba_b7b8_4fbfe3c87fa9.slice/crio-090ffae72301c4b6458ea6846a5db846afd82dfd67ed273dcad570a7ae20190a WatchSource:0}: Error finding container 090ffae72301c4b6458ea6846a5db846afd82dfd67ed273dcad570a7ae20190a: Status 404 returned error can't find the container with id 090ffae72301c4b6458ea6846a5db846afd82dfd67ed273dcad570a7ae20190a Jul 24 13:56:14.881838 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:14.878428 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa6e6fd8-f625-46e2-be3a-880646f7781a" path="/var/lib/kubelet/pods/aa6e6fd8-f625-46e2-be3a-880646f7781a/volumes" Jul 24 13:56:15.149232 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:15.149144 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f7f8bcdc-wcq4d" event={"ID":"6cf7b06e-53ff-49ba-b7b8-4fbfe3c87fa9","Type":"ContainerStarted","Data":"2a704d77b3c8c4838817bc6635805d7fe6f7c227afb12c696ec08abcbf8f6444"} Jul 24 13:56:15.149232 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:15.149196 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f7f8bcdc-wcq4d" event={"ID":"6cf7b06e-53ff-49ba-b7b8-4fbfe3c87fa9","Type":"ContainerStarted","Data":"090ffae72301c4b6458ea6846a5db846afd82dfd67ed273dcad570a7ae20190a"} Jul 24 13:56:15.149635 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:15.149292 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76f7f8bcdc-wcq4d" Jul 24 13:56:15.169232 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:15.169168 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76f7f8bcdc-wcq4d" podStartSLOduration=1.7650182779999999 podStartE2EDuration="2.169152456s" totalImagesPullingTime="404.134178ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:56:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:56:15.167267727 +0000 UTC m=+808.854029148" watchObservedRunningTime="2026-07-24 13:56:15.169152456 +0000 UTC m=+808.855913876" Jul 24 13:56:26.155940 ip-10-0-128-250 kubenswrapper[2557]: I0724 13:56:26.155909 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76f7f8bcdc-wcq4d" Jul 24 14:04:28.194541 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.194509 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv"] Jul 24 14:04:28.197892 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.197874 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.200099 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.200074 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-080e706b-kserve-self-signed-certs\"" Jul 24 14:04:28.216052 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.215999 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv"] Jul 24 14:04:28.334193 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.334154 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d82l\" (UniqueName: \"kubernetes.io/projected/23584e29-ceb3-4c74-bf67-3ce644c83189-kube-api-access-9d82l\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.334193 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.334190 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-home\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.334549 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.334258 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-dshm\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.334549 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.334312 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23584e29-ceb3-4c74-bf67-3ce644c83189-tls-certs\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.334549 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.334333 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-tmp-dir\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.334549 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.334352 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-model-cache\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.435622 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.435577 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d82l\" (UniqueName: \"kubernetes.io/projected/23584e29-ceb3-4c74-bf67-3ce644c83189-kube-api-access-9d82l\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.435622 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.435618 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-home\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.435923 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.435649 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-dshm\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.435923 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.435671 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23584e29-ceb3-4c74-bf67-3ce644c83189-tls-certs\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.435923 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.435692 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-tmp-dir\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.435923 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.435859 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-model-cache\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.436150 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.436059 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-home\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.436150 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.436118 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-tmp-dir\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.436226 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.436184 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-model-cache\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.438249 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.438222 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-dshm\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.438400 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.438381 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23584e29-ceb3-4c74-bf67-3ce644c83189-tls-certs\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.443899 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.443878 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d82l\" (UniqueName: \"kubernetes.io/projected/23584e29-ceb3-4c74-bf67-3ce644c83189-kube-api-access-9d82l\") pod \"e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.507670 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.507587 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:28.642522 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:28.642498 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv"] Jul 24 14:04:28.644650 ip-10-0-128-250 kubenswrapper[2557]: W0724 14:04:28.644621 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23584e29_ceb3_4c74_bf67_3ce644c83189.slice/crio-1cded2806853aef105b1a78fdefc2795ab1f6308cca88435ae6604bf2da38b99 WatchSource:0}: Error finding container 1cded2806853aef105b1a78fdefc2795ab1f6308cca88435ae6604bf2da38b99: Status 404 returned error can't find the container with id 1cded2806853aef105b1a78fdefc2795ab1f6308cca88435ae6604bf2da38b99 Jul 24 14:04:29.208629 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:29.208590 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" event={"ID":"23584e29-ceb3-4c74-bf67-3ce644c83189","Type":"ContainerStarted","Data":"d8736a6b7d6d3528e41f5f79d19c4bf8d74baa65bb27a7b089b5f778a4893fed"} Jul 24 14:04:29.209111 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:29.208640 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" event={"ID":"23584e29-ceb3-4c74-bf67-3ce644c83189","Type":"ContainerStarted","Data":"1cded2806853aef105b1a78fdefc2795ab1f6308cca88435ae6604bf2da38b99"} Jul 24 14:04:29.226038 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:29.225978 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" podStartSLOduration=1.225963629 podStartE2EDuration="1.225963629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:04:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:04:29.224974037 +0000 UTC m=+1302.911735454" watchObservedRunningTime="2026-07-24 14:04:29.225963629 +0000 UTC m=+1302.912725045" Jul 24 14:04:34.973595 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:34.973557 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25"] Jul 24 14:04:34.977296 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:34.977268 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:34.979844 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:34.979800 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-080e706b-kserve-self-signed-certs\"" Jul 24 14:04:34.988793 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:34.988766 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25"] Jul 24 14:04:35.091063 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.091031 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-model-cache\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.091237 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.091072 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f70097f8-5236-41ca-bceb-c9560622e3e2-tls-certs\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.091237 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.091144 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl57z\" (UniqueName: \"kubernetes.io/projected/f70097f8-5236-41ca-bceb-c9560622e3e2-kube-api-access-pl57z\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.091237 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.091164 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-home\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.091356 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.091329 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-dshm\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.091400 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.091374 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-tmp-dir\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.192720 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.192686 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-model-cache\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.192957 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.192730 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f70097f8-5236-41ca-bceb-c9560622e3e2-tls-certs\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.192957 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.192776 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl57z\" (UniqueName: \"kubernetes.io/projected/f70097f8-5236-41ca-bceb-c9560622e3e2-kube-api-access-pl57z\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.192957 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.192794 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-home\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.192957 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.192870 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-dshm\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.192957 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.192889 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-tmp-dir\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.193225 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.193182 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-model-cache\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.193225 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.193196 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-home\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.193329 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.193250 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-tmp-dir\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.195351 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.195325 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-dshm\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.196009 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.195622 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f70097f8-5236-41ca-bceb-c9560622e3e2-tls-certs\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.205072 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.205045 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl57z\" (UniqueName: \"kubernetes.io/projected/f70097f8-5236-41ca-bceb-c9560622e3e2-kube-api-access-pl57z\") pod \"e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.287450 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.287344 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:35.428798 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:35.426535 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25"] Jul 24 14:04:35.429606 ip-10-0-128-250 kubenswrapper[2557]: W0724 14:04:35.429579 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf70097f8_5236_41ca_bceb_c9560622e3e2.slice/crio-f96cbd4307896b0560c4dca392dce46745ebbabef0c4dddf0e193cefce618204 WatchSource:0}: Error finding container f96cbd4307896b0560c4dca392dce46745ebbabef0c4dddf0e193cefce618204: Status 404 returned error can't find the container with id f96cbd4307896b0560c4dca392dce46745ebbabef0c4dddf0e193cefce618204 Jul 24 14:04:36.239392 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:36.239356 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" event={"ID":"f70097f8-5236-41ca-bceb-c9560622e3e2","Type":"ContainerStarted","Data":"e07f12230bc5c740cb411f818d4b2a42fe96d836a4fa4a39de65b0bcb8484750"} Jul 24 14:04:36.239779 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:36.239409 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" event={"ID":"f70097f8-5236-41ca-bceb-c9560622e3e2","Type":"ContainerStarted","Data":"f96cbd4307896b0560c4dca392dce46745ebbabef0c4dddf0e193cefce618204"} Jul 24 14:04:36.254229 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:36.254181 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" podStartSLOduration=2.254166996 podStartE2EDuration="2.254166996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:04:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:04:36.252839521 +0000 UTC m=+1309.939600932" watchObservedRunningTime="2026-07-24 14:04:36.254166996 +0000 UTC m=+1309.940928414" Jul 24 14:04:38.507899 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:38.507788 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:38.507899 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:38.507853 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:38.520469 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:38.520441 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:39.262841 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:39.262793 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:04:45.288205 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:45.288166 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:45.288852 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:45.288223 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:45.300866 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:45.300843 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:04:46.293134 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:04:46.293101 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:05:03.052695 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:03.052662 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv"] Jul 24 14:05:04.652247 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:04.652214 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25"] Jul 24 14:05:18.073015 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.072903 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" podUID="23584e29-ceb3-4c74-bf67-3ce644c83189" containerName="main" containerID="cri-o://d8736a6b7d6d3528e41f5f79d19c4bf8d74baa65bb27a7b089b5f778a4893fed" gracePeriod=45 Jul 24 14:05:18.317739 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.317714 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:05:18.420190 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.420155 2557 generic.go:356] "Generic (PLEG): container finished" podID="23584e29-ceb3-4c74-bf67-3ce644c83189" containerID="d8736a6b7d6d3528e41f5f79d19c4bf8d74baa65bb27a7b089b5f778a4893fed" exitCode=0 Jul 24 14:05:18.420383 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.420202 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" Jul 24 14:05:18.420383 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.420240 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" event={"ID":"23584e29-ceb3-4c74-bf67-3ce644c83189","Type":"ContainerDied","Data":"d8736a6b7d6d3528e41f5f79d19c4bf8d74baa65bb27a7b089b5f778a4893fed"} Jul 24 14:05:18.420383 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.420290 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv" event={"ID":"23584e29-ceb3-4c74-bf67-3ce644c83189","Type":"ContainerDied","Data":"1cded2806853aef105b1a78fdefc2795ab1f6308cca88435ae6604bf2da38b99"} Jul 24 14:05:18.420383 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.420309 2557 scope.go:117] "RemoveContainer" containerID="d8736a6b7d6d3528e41f5f79d19c4bf8d74baa65bb27a7b089b5f778a4893fed" Jul 24 14:05:18.428664 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.428647 2557 scope.go:117] "RemoveContainer" containerID="d8736a6b7d6d3528e41f5f79d19c4bf8d74baa65bb27a7b089b5f778a4893fed" Jul 24 14:05:18.428968 ip-10-0-128-250 kubenswrapper[2557]: E0724 14:05:18.428946 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8736a6b7d6d3528e41f5f79d19c4bf8d74baa65bb27a7b089b5f778a4893fed\": container with ID starting with d8736a6b7d6d3528e41f5f79d19c4bf8d74baa65bb27a7b089b5f778a4893fed not found: ID does not exist" containerID="d8736a6b7d6d3528e41f5f79d19c4bf8d74baa65bb27a7b089b5f778a4893fed" Jul 24 14:05:18.429076 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.428975 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8736a6b7d6d3528e41f5f79d19c4bf8d74baa65bb27a7b089b5f778a4893fed"} err="failed to get container status \"d8736a6b7d6d3528e41f5f79d19c4bf8d74baa65bb27a7b089b5f778a4893fed\": rpc error: code = NotFound desc = could not find container \"d8736a6b7d6d3528e41f5f79d19c4bf8d74baa65bb27a7b089b5f778a4893fed\": container with ID starting with d8736a6b7d6d3528e41f5f79d19c4bf8d74baa65bb27a7b089b5f778a4893fed not found: ID does not exist" Jul 24 14:05:18.470908 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.470873 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-dshm\") pod \"23584e29-ceb3-4c74-bf67-3ce644c83189\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " Jul 24 14:05:18.471082 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.470927 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-model-cache\") pod \"23584e29-ceb3-4c74-bf67-3ce644c83189\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " Jul 24 14:05:18.471082 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.470956 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9d82l\" (UniqueName: \"kubernetes.io/projected/23584e29-ceb3-4c74-bf67-3ce644c83189-kube-api-access-9d82l\") pod \"23584e29-ceb3-4c74-bf67-3ce644c83189\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " Jul 24 14:05:18.471082 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.470993 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-home\") pod \"23584e29-ceb3-4c74-bf67-3ce644c83189\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " Jul 24 14:05:18.471082 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.471063 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23584e29-ceb3-4c74-bf67-3ce644c83189-tls-certs\") pod \"23584e29-ceb3-4c74-bf67-3ce644c83189\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " Jul 24 14:05:18.471339 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.471139 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-tmp-dir\") pod \"23584e29-ceb3-4c74-bf67-3ce644c83189\" (UID: \"23584e29-ceb3-4c74-bf67-3ce644c83189\") " Jul 24 14:05:18.471339 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.471245 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-model-cache" (OuterVolumeSpecName: "model-cache") pod "23584e29-ceb3-4c74-bf67-3ce644c83189" (UID: "23584e29-ceb3-4c74-bf67-3ce644c83189"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:05:18.471339 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.471321 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-home" (OuterVolumeSpecName: "home") pod "23584e29-ceb3-4c74-bf67-3ce644c83189" (UID: "23584e29-ceb3-4c74-bf67-3ce644c83189"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:05:18.471474 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.471431 2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-model-cache\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 14:05:18.471510 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.471480 2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-home\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 14:05:18.471510 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.471479 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "23584e29-ceb3-4c74-bf67-3ce644c83189" (UID: "23584e29-ceb3-4c74-bf67-3ce644c83189"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:05:18.473377 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.473341 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23584e29-ceb3-4c74-bf67-3ce644c83189-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "23584e29-ceb3-4c74-bf67-3ce644c83189" (UID: "23584e29-ceb3-4c74-bf67-3ce644c83189"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 14:05:18.473649 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.473628 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-dshm" (OuterVolumeSpecName: "dshm") pod "23584e29-ceb3-4c74-bf67-3ce644c83189" (UID: "23584e29-ceb3-4c74-bf67-3ce644c83189"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:05:18.473745 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.473652 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23584e29-ceb3-4c74-bf67-3ce644c83189-kube-api-access-9d82l" (OuterVolumeSpecName: "kube-api-access-9d82l") pod "23584e29-ceb3-4c74-bf67-3ce644c83189" (UID: "23584e29-ceb3-4c74-bf67-3ce644c83189"). InnerVolumeSpecName "kube-api-access-9d82l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:05:18.572036 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.572000 2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23584e29-ceb3-4c74-bf67-3ce644c83189-tls-certs\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 14:05:18.572036 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.572028 2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-tmp-dir\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 14:05:18.572036 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.572042 2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23584e29-ceb3-4c74-bf67-3ce644c83189-dshm\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 14:05:18.572271 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.572053 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d82l\" (UniqueName: \"kubernetes.io/projected/23584e29-ceb3-4c74-bf67-3ce644c83189-kube-api-access-9d82l\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 14:05:18.741698 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.741669 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv"] Jul 24 14:05:18.744974 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.744947 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-080e706b-kserve-64bd59f7c9-cm2sv"] Jul 24 14:05:18.876971 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:18.876943 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23584e29-ceb3-4c74-bf67-3ce644c83189" path="/var/lib/kubelet/pods/23584e29-ceb3-4c74-bf67-3ce644c83189/volumes" Jul 24 14:05:19.672711 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:19.672665 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" podUID="f70097f8-5236-41ca-bceb-c9560622e3e2" containerName="main" containerID="cri-o://e07f12230bc5c740cb411f818d4b2a42fe96d836a4fa4a39de65b0bcb8484750" gracePeriod=45 Jul 24 14:05:19.930366 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:19.930303 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:05:20.084090 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.084057 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-model-cache\") pod \"f70097f8-5236-41ca-bceb-c9560622e3e2\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " Jul 24 14:05:20.084292 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.084100 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pl57z\" (UniqueName: \"kubernetes.io/projected/f70097f8-5236-41ca-bceb-c9560622e3e2-kube-api-access-pl57z\") pod \"f70097f8-5236-41ca-bceb-c9560622e3e2\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " Jul 24 14:05:20.084292 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.084119 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-tmp-dir\") pod \"f70097f8-5236-41ca-bceb-c9560622e3e2\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " Jul 24 14:05:20.084292 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.084175 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f70097f8-5236-41ca-bceb-c9560622e3e2-tls-certs\") pod \"f70097f8-5236-41ca-bceb-c9560622e3e2\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " Jul 24 14:05:20.084292 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.084223 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-dshm\") pod \"f70097f8-5236-41ca-bceb-c9560622e3e2\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " Jul 24 14:05:20.084292 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.084280 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-home\") pod \"f70097f8-5236-41ca-bceb-c9560622e3e2\" (UID: \"f70097f8-5236-41ca-bceb-c9560622e3e2\") " Jul 24 14:05:20.084564 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.084378 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-model-cache" (OuterVolumeSpecName: "model-cache") pod "f70097f8-5236-41ca-bceb-c9560622e3e2" (UID: "f70097f8-5236-41ca-bceb-c9560622e3e2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:05:20.084564 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.084400 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f70097f8-5236-41ca-bceb-c9560622e3e2" (UID: "f70097f8-5236-41ca-bceb-c9560622e3e2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:05:20.084673 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.084599 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-home" (OuterVolumeSpecName: "home") pod "f70097f8-5236-41ca-bceb-c9560622e3e2" (UID: "f70097f8-5236-41ca-bceb-c9560622e3e2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:05:20.084673 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.084603 2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-model-cache\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 14:05:20.084673 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.084630 2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-tmp-dir\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 14:05:20.086438 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.086416 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-dshm" (OuterVolumeSpecName: "dshm") pod "f70097f8-5236-41ca-bceb-c9560622e3e2" (UID: "f70097f8-5236-41ca-bceb-c9560622e3e2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:05:20.086438 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.086426 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f70097f8-5236-41ca-bceb-c9560622e3e2-kube-api-access-pl57z" (OuterVolumeSpecName: "kube-api-access-pl57z") pod "f70097f8-5236-41ca-bceb-c9560622e3e2" (UID: "f70097f8-5236-41ca-bceb-c9560622e3e2"). InnerVolumeSpecName "kube-api-access-pl57z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:05:20.086579 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.086562 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f70097f8-5236-41ca-bceb-c9560622e3e2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f70097f8-5236-41ca-bceb-c9560622e3e2" (UID: "f70097f8-5236-41ca-bceb-c9560622e3e2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 14:05:20.186093 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.186004 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pl57z\" (UniqueName: \"kubernetes.io/projected/f70097f8-5236-41ca-bceb-c9560622e3e2-kube-api-access-pl57z\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 14:05:20.186093 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.186034 2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f70097f8-5236-41ca-bceb-c9560622e3e2-tls-certs\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 14:05:20.186093 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.186047 2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-dshm\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 14:05:20.186093 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.186054 2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f70097f8-5236-41ca-bceb-c9560622e3e2-home\") on node \"ip-10-0-128-250.ec2.internal\" DevicePath \"\"" Jul 24 14:05:20.430898 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.430859 2557 generic.go:356] "Generic (PLEG): container finished" podID="f70097f8-5236-41ca-bceb-c9560622e3e2" containerID="e07f12230bc5c740cb411f818d4b2a42fe96d836a4fa4a39de65b0bcb8484750" exitCode=0 Jul 24 14:05:20.431097 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.430908 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" Jul 24 14:05:20.431097 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.430948 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" event={"ID":"f70097f8-5236-41ca-bceb-c9560622e3e2","Type":"ContainerDied","Data":"e07f12230bc5c740cb411f818d4b2a42fe96d836a4fa4a39de65b0bcb8484750"} Jul 24 14:05:20.431097 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.431005 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25" event={"ID":"f70097f8-5236-41ca-bceb-c9560622e3e2","Type":"ContainerDied","Data":"f96cbd4307896b0560c4dca392dce46745ebbabef0c4dddf0e193cefce618204"} Jul 24 14:05:20.431097 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.431027 2557 scope.go:117] "RemoveContainer" containerID="e07f12230bc5c740cb411f818d4b2a42fe96d836a4fa4a39de65b0bcb8484750" Jul 24 14:05:20.439673 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.439653 2557 scope.go:117] "RemoveContainer" containerID="e07f12230bc5c740cb411f818d4b2a42fe96d836a4fa4a39de65b0bcb8484750" Jul 24 14:05:20.439936 ip-10-0-128-250 kubenswrapper[2557]: E0724 14:05:20.439919 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e07f12230bc5c740cb411f818d4b2a42fe96d836a4fa4a39de65b0bcb8484750\": container with ID starting with e07f12230bc5c740cb411f818d4b2a42fe96d836a4fa4a39de65b0bcb8484750 not found: ID does not exist" containerID="e07f12230bc5c740cb411f818d4b2a42fe96d836a4fa4a39de65b0bcb8484750" Jul 24 14:05:20.440024 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.439945 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e07f12230bc5c740cb411f818d4b2a42fe96d836a4fa4a39de65b0bcb8484750"} err="failed to get container status \"e07f12230bc5c740cb411f818d4b2a42fe96d836a4fa4a39de65b0bcb8484750\": rpc error: code = NotFound desc = could not find container \"e07f12230bc5c740cb411f818d4b2a42fe96d836a4fa4a39de65b0bcb8484750\": container with ID starting with e07f12230bc5c740cb411f818d4b2a42fe96d836a4fa4a39de65b0bcb8484750 not found: ID does not exist" Jul 24 14:05:20.451148 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.451118 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25"] Jul 24 14:05:20.456741 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.456716 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-080e706b-kserve-7df5d8d855-8gm25"] Jul 24 14:05:20.878728 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:05:20.878698 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f70097f8-5236-41ca-bceb-c9560622e3e2" path="/var/lib/kubelet/pods/f70097f8-5236-41ca-bceb-c9560622e3e2/volumes" Jul 24 14:08:11.335958 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:11.335888 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76f7f8bcdc-wcq4d_6cf7b06e-53ff-49ba-b7b8-4fbfe3c87fa9/manager/0.log" Jul 24 14:08:11.585300 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:11.585276 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-6fftq_bb84f68e-d4ae-43c2-bc54-33c496c21c71/manager/0.log" Jul 24 14:08:12.012104 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:12.012066 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-674cffbf7-snrf8_a737c1d9-5044-4cb2-a972-d206ce0c0df5/manager/0.log" Jul 24 14:08:12.025070 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:12.025050 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-674cffbf7-snrf8_a737c1d9-5044-4cb2-a972-d206ce0c0df5/copy-manifests/0.log" Jul 24 14:08:12.421357 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:12.421331 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-95d8b655f-2snl5_5bd69357-578e-4bd2-8028-6369445e23ad/maas-api/0.log" Jul 24 14:08:15.397508 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:15.397477 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z_8d222f02-d311-4503-be99-7c1ec3043ac5/istio-proxy/0.log" Jul 24 14:08:15.639195 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:15.639168 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c9467694d-hhcmz_0f322fa1-0998-4846-b50e-436ae1949227/kube-auth-proxy/0.log" Jul 24 14:08:16.692002 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:16.691970 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-lcgdm_f3060d59-d4a8-4638-a7bb-824c9b8ca918/main/0.log" Jul 24 14:08:17.045887 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:17.045798 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-x26sj_c4cac94b-46e7-4ac1-a3c5-8025350837d8/main/0.log" Jul 24 14:08:17.157924 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:17.157897 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-2zh9q_7c48264f-fd02-416b-a63f-98944fce2771/main/0.log" Jul 24 14:08:25.722291 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:25.722261 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-q7hvp_533b42cc-ae61-4729-b5a1-e904f90f7fcc/global-pull-secret-syncer/0.log" Jul 24 14:08:25.793462 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:25.793419 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tqkzp_3f4a0f2f-93e0-40f6-9ec7-6b61a09ade21/konnectivity-agent/0.log" Jul 24 14:08:25.837608 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:25.837582 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-250.ec2.internal_c126e9785f9c1e44eb018143a4d59ea2/haproxy/0.log" Jul 24 14:08:33.292582 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:33.292547 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-npd7p_a88ae537-f807-4aa1-ab8e-da01013105ad/node-exporter/0.log" Jul 24 14:08:33.309322 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:33.309298 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-npd7p_a88ae537-f807-4aa1-ab8e-da01013105ad/kube-rbac-proxy/0.log" Jul 24 14:08:33.325270 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:33.325242 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-npd7p_a88ae537-f807-4aa1-ab8e-da01013105ad/init-textfile/0.log" Jul 24 14:08:34.917904 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:34.917875 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-hhx58_baf4fefa-586a-4887-b3ce-9fd95f191b83/networking-console-plugin/0.log" Jul 24 14:08:35.150796 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.150764 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb"] Jul 24 14:08:35.151099 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.151087 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23584e29-ceb3-4c74-bf67-3ce644c83189" containerName="main" Jul 24 14:08:35.151148 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.151101 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="23584e29-ceb3-4c74-bf67-3ce644c83189" containerName="main" Jul 24 14:08:35.151148 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.151110 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f70097f8-5236-41ca-bceb-c9560622e3e2" containerName="main" Jul 24 14:08:35.151148 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.151115 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f70097f8-5236-41ca-bceb-c9560622e3e2" containerName="main" Jul 24 14:08:35.151238 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.151181 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="23584e29-ceb3-4c74-bf67-3ce644c83189" containerName="main" Jul 24 14:08:35.151238 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.151187 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f70097f8-5236-41ca-bceb-c9560622e3e2" containerName="main" Jul 24 14:08:35.154115 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.154099 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.156146 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.156117 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t2qd9\"/\"default-dockercfg-2jftm\"" Jul 24 14:08:35.156275 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.156148 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t2qd9\"/\"openshift-service-ca.crt\"" Jul 24 14:08:35.156705 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.156690 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t2qd9\"/\"kube-root-ca.crt\"" Jul 24 14:08:35.163148 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.163124 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb"] Jul 24 14:08:35.262629 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.262541 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmtqk\" (UniqueName: \"kubernetes.io/projected/6019cdac-3659-435e-a2e8-28c980019f30-kube-api-access-mmtqk\") pod \"perf-node-gather-daemonset-cszhb\" (UID: \"6019cdac-3659-435e-a2e8-28c980019f30\") " pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.262629 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.262583 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6019cdac-3659-435e-a2e8-28c980019f30-sys\") pod \"perf-node-gather-daemonset-cszhb\" (UID: \"6019cdac-3659-435e-a2e8-28c980019f30\") " pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.262629 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.262605 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6019cdac-3659-435e-a2e8-28c980019f30-podres\") pod \"perf-node-gather-daemonset-cszhb\" (UID: \"6019cdac-3659-435e-a2e8-28c980019f30\") " pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.262880 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.262735 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6019cdac-3659-435e-a2e8-28c980019f30-proc\") pod \"perf-node-gather-daemonset-cszhb\" (UID: \"6019cdac-3659-435e-a2e8-28c980019f30\") " pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.262880 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.262766 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6019cdac-3659-435e-a2e8-28c980019f30-lib-modules\") pod \"perf-node-gather-daemonset-cszhb\" (UID: \"6019cdac-3659-435e-a2e8-28c980019f30\") " pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.364055 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.364017 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6019cdac-3659-435e-a2e8-28c980019f30-podres\") pod \"perf-node-gather-daemonset-cszhb\" (UID: \"6019cdac-3659-435e-a2e8-28c980019f30\") " pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.364236 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.364092 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6019cdac-3659-435e-a2e8-28c980019f30-proc\") pod \"perf-node-gather-daemonset-cszhb\" (UID: \"6019cdac-3659-435e-a2e8-28c980019f30\") " pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.364236 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.364110 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6019cdac-3659-435e-a2e8-28c980019f30-lib-modules\") pod \"perf-node-gather-daemonset-cszhb\" (UID: \"6019cdac-3659-435e-a2e8-28c980019f30\") " pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.364236 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.364142 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmtqk\" (UniqueName: \"kubernetes.io/projected/6019cdac-3659-435e-a2e8-28c980019f30-kube-api-access-mmtqk\") pod \"perf-node-gather-daemonset-cszhb\" (UID: \"6019cdac-3659-435e-a2e8-28c980019f30\") " pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.364236 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.364163 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6019cdac-3659-435e-a2e8-28c980019f30-sys\") pod \"perf-node-gather-daemonset-cszhb\" (UID: \"6019cdac-3659-435e-a2e8-28c980019f30\") " pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.364236 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.364225 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6019cdac-3659-435e-a2e8-28c980019f30-sys\") pod \"perf-node-gather-daemonset-cszhb\" (UID: \"6019cdac-3659-435e-a2e8-28c980019f30\") " pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.364236 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.364224 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6019cdac-3659-435e-a2e8-28c980019f30-podres\") pod \"perf-node-gather-daemonset-cszhb\" (UID: \"6019cdac-3659-435e-a2e8-28c980019f30\") " pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.364530 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.364224 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6019cdac-3659-435e-a2e8-28c980019f30-proc\") pod \"perf-node-gather-daemonset-cszhb\" (UID: \"6019cdac-3659-435e-a2e8-28c980019f30\") " pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.364530 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.364278 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6019cdac-3659-435e-a2e8-28c980019f30-lib-modules\") pod \"perf-node-gather-daemonset-cszhb\" (UID: \"6019cdac-3659-435e-a2e8-28c980019f30\") " pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.377551 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.374908 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmtqk\" (UniqueName: \"kubernetes.io/projected/6019cdac-3659-435e-a2e8-28c980019f30-kube-api-access-mmtqk\") pod \"perf-node-gather-daemonset-cszhb\" (UID: \"6019cdac-3659-435e-a2e8-28c980019f30\") " pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.464338 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.464287 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:35.596931 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:35.596898 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb"] Jul 24 14:08:35.599370 ip-10-0-128-250 kubenswrapper[2557]: W0724 14:08:35.599339 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6019cdac_3659_435e_a2e8_28c980019f30.slice/crio-539e2d42a86abaf556a43ca1e3afc6d0753fa0780a76ffc5b4d1cc3d17a19064 WatchSource:0}: Error finding container 539e2d42a86abaf556a43ca1e3afc6d0753fa0780a76ffc5b4d1cc3d17a19064: Status 404 returned error can't find the container with id 539e2d42a86abaf556a43ca1e3afc6d0753fa0780a76ffc5b4d1cc3d17a19064 Jul 24 14:08:36.236007 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:36.235974 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" event={"ID":"6019cdac-3659-435e-a2e8-28c980019f30","Type":"ContainerStarted","Data":"5d0908e15f768a07a1f596956069da9cd11a448564930b31e0b5beecd7077cdf"} Jul 24 14:08:36.236447 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:36.236034 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" event={"ID":"6019cdac-3659-435e-a2e8-28c980019f30","Type":"ContainerStarted","Data":"539e2d42a86abaf556a43ca1e3afc6d0753fa0780a76ffc5b4d1cc3d17a19064"} Jul 24 14:08:36.236447 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:36.236062 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:36.252485 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:36.252428 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" podStartSLOduration=1.2524126230000001 podStartE2EDuration="1.252412623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:08:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:08:36.251393901 +0000 UTC m=+1549.938155318" watchObservedRunningTime="2026-07-24 14:08:36.252412623 +0000 UTC m=+1549.939174053" Jul 24 14:08:37.611087 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:37.611056 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h2vp7_9e7565d1-c59f-4fd9-adcd-09042dd2345e/dns/0.log" Jul 24 14:08:37.639211 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:37.639180 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h2vp7_9e7565d1-c59f-4fd9-adcd-09042dd2345e/kube-rbac-proxy/0.log" Jul 24 14:08:37.698967 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:37.698938 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5mnw5_5513d77f-a9df-4125-bd63-d8f9df7b7e93/dns-node-resolver/0.log" Jul 24 14:08:38.213287 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:38.213255 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7fb94d7fbb-6kwxh_65600164-6849-4bb2-83c2-1be903b7b59a/registry/0.log" Jul 24 14:08:38.230731 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:38.230706 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j5cfd_9df22139-a68a-421b-807c-e6afb4bb691e/node-ca/0.log" Jul 24 14:08:39.161008 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:39.160976 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z_8d222f02-d311-4503-be99-7c1ec3043ac5/istio-proxy/0.log" Jul 24 14:08:39.231056 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:39.231023 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c9467694d-hhcmz_0f322fa1-0998-4846-b50e-436ae1949227/kube-auth-proxy/0.log" Jul 24 14:08:40.062219 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:40.062192 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-25lb7_849e0755-ade6-4136-9394-017410ace217/serve-healthcheck-canary/0.log" Jul 24 14:08:40.683502 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:40.683477 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fvvgk_3d96004d-7277-4a7e-9ec7-a87faa9526fd/kube-rbac-proxy/0.log" Jul 24 14:08:40.702329 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:40.702305 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fvvgk_3d96004d-7277-4a7e-9ec7-a87faa9526fd/exporter/0.log" Jul 24 14:08:40.720368 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:40.720337 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fvvgk_3d96004d-7277-4a7e-9ec7-a87faa9526fd/extractor/0.log" Jul 24 14:08:42.250623 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:42.250596 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t2qd9/perf-node-gather-daemonset-cszhb" Jul 24 14:08:42.971215 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:42.971183 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76f7f8bcdc-wcq4d_6cf7b06e-53ff-49ba-b7b8-4fbfe3c87fa9/manager/0.log" Jul 24 14:08:43.035212 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:43.035175 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-6fftq_bb84f68e-d4ae-43c2-bc54-33c496c21c71/manager/0.log" Jul 24 14:08:43.177351 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:43.177319 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-674cffbf7-snrf8_a737c1d9-5044-4cb2-a972-d206ce0c0df5/manager/0.log" Jul 24 14:08:43.191248 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:43.191220 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-674cffbf7-snrf8_a737c1d9-5044-4cb2-a972-d206ce0c0df5/copy-manifests/0.log" Jul 24 14:08:44.502214 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:44.502123 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78d5cc8c46-pvcx5_c87ff2ec-0cf3-4aa1-8e2a-cc51176a3d54/manager/0.log" Jul 24 14:08:45.129718 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:45.129692 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z_8d222f02-d311-4503-be99-7c1ec3043ac5/istio-proxy/0.log" Jul 24 14:08:45.187609 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:45.187583 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c9467694d-hhcmz_0f322fa1-0998-4846-b50e-436ae1949227/kube-auth-proxy/0.log" Jul 24 14:08:46.260778 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:46.260752 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-cgm6z_8d222f02-d311-4503-be99-7c1ec3043ac5/istio-proxy/0.log" Jul 24 14:08:46.320326 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:46.320285 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c9467694d-hhcmz_0f322fa1-0998-4846-b50e-436ae1949227/kube-auth-proxy/0.log" Jul 24 14:08:49.385474 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:49.385450 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-brq2q_0e3799ec-fba5-4ec2-b848-495b192761e8/migrator/0.log" Jul 24 14:08:49.401503 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:49.401471 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-brq2q_0e3799ec-fba5-4ec2-b848-495b192761e8/graceful-termination/0.log" Jul 24 14:08:50.838339 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:50.838308 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lmwnz_6df914a7-aff6-4e3a-936b-a8852546642f/kube-multus-additional-cni-plugins/0.log" Jul 24 14:08:50.858099 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:50.858075 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lmwnz_6df914a7-aff6-4e3a-936b-a8852546642f/egress-router-binary-copy/0.log" Jul 24 14:08:50.877103 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:50.877074 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lmwnz_6df914a7-aff6-4e3a-936b-a8852546642f/cni-plugins/0.log" Jul 24 14:08:50.900421 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:50.900398 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lmwnz_6df914a7-aff6-4e3a-936b-a8852546642f/bond-cni-plugin/0.log" Jul 24 14:08:50.918599 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:50.918568 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lmwnz_6df914a7-aff6-4e3a-936b-a8852546642f/routeoverride-cni/0.log" Jul 24 14:08:50.938174 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:50.938148 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lmwnz_6df914a7-aff6-4e3a-936b-a8852546642f/whereabouts-cni-bincopy/0.log" Jul 24 14:08:50.959370 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:50.959292 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lmwnz_6df914a7-aff6-4e3a-936b-a8852546642f/whereabouts-cni/0.log" Jul 24 14:08:51.129964 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:51.129936 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fdrqb_c799cb15-d6c2-4140-9af0-64b652841355/kube-multus/0.log" Jul 24 14:08:51.269247 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:51.269221 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vjcfb_46030f6f-52fc-40f7-b5d4-7febe8bd998a/network-metrics-daemon/0.log" Jul 24 14:08:51.284363 ip-10-0-128-250 kubenswrapper[2557]: I0724 14:08:51.284326 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vjcfb_46030f6f-52fc-40f7-b5d4-7febe8bd998a/kube-rbac-proxy/0.log"