Sep 10 18:31:09.660389 ip-10-0-138-87 systemd[1]: Starting Kubernetes Kubelet... Sep 10 18:31:10.112352 ip-10-0-138-87 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 18:31:10.112352 ip-10-0-138-87 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 18:31:10.112352 ip-10-0-138-87 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 18:31:10.112352 ip-10-0-138-87 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 18:31:10.112352 ip-10-0-138-87 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 18:31:10.113064 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.112433 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 18:31:10.116551 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116535 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 18:31:10.116551 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116550 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116554 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116558 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116561 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116564 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116566 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116569 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116572 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116574 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116578 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116580 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116583 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116587 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116591 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116594 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116597 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116600 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116603 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116606 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 18:31:10.116614 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116613 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116616 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116618 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116621 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116625 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116627 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116630 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116632 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116635 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116638 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116641 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116643 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116646 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116651 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116654 2564 feature_gate.go:328] unrecognized feature gate: Example Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116658 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116661 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116663 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116666 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116668 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 18:31:10.117096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116671 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116673 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116676 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116678 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116681 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116683 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116686 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116688 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116691 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116693 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116696 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116699 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116703 2564 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116706 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116710 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116713 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116716 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116719 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116721 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116724 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 18:31:10.117582 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116727 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116730 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116732 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116735 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116737 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116741 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116744 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116746 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116749 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116751 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116754 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116756 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116759 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116761 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116764 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116766 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116769 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116771 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116776 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116778 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 18:31:10.118139 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116781 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116784 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116786 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116789 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116791 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116796 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116799 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116802 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116805 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116807 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116810 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116812 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116815 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116818 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116820 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116822 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116825 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116828 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.116830 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117551 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 18:31:10.118637 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117558 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117562 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117565 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117568 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117571 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117574 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117578 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117582 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117586 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117589 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117592 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117595 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117598 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117601 2564 feature_gate.go:328] unrecognized feature gate: Example Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117604 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117606 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117609 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117613 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117616 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117619 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 18:31:10.119142 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117622 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117624 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117627 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117630 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117633 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117635 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117638 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117640 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117643 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117645 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117648 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117651 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117653 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117656 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117658 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117661 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117664 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117666 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117669 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117671 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 18:31:10.119641 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117673 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117676 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117678 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117681 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117683 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117686 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117689 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117691 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117694 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117698 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117702 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117705 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117708 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117711 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117713 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117715 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117718 2564 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117720 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117723 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117726 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 18:31:10.120143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117729 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117731 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117733 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117736 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117739 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117741 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117744 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117747 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117750 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117753 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117755 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117758 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117761 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117764 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117767 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117769 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117772 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117775 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117778 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117780 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 18:31:10.120639 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117783 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117785 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117788 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117790 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117793 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117796 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117799 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117801 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117804 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117806 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117809 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117812 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117816 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117819 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117822 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117828 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117830 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.117833 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118240 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 18:31:10.121143 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118245 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118248 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118251 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118253 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118256 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118259 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118261 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118264 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118267 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118269 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118272 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118275 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118278 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118281 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118283 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118286 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118288 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118291 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118294 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118297 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 18:31:10.121606 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118300 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118302 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118305 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118308 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118310 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118313 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118315 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118318 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118324 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118326 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118329 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118331 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118334 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118336 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118339 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118342 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118344 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118347 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118349 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118351 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 18:31:10.122120 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118354 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118357 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118360 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118362 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118365 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118367 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118370 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118372 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118375 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118378 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118380 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118384 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118387 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118390 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118392 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118395 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118397 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118400 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118402 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 18:31:10.122615 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118405 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118409 2564 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118411 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118414 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118416 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118419 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118423 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118427 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118430 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118433 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118436 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118438 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118441 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118444 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118446 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118449 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118452 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118454 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118456 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 18:31:10.123122 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118459 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118461 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118464 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118466 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118469 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118471 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118476 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118479 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118482 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118484 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118487 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118489 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118492 2564 feature_gate.go:328] unrecognized feature gate: Example Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118494 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118498 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118500 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118503 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118505 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118508 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.118512 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 18:31:10.123586 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118583 2564 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118615 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118622 2564 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118626 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118630 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118634 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118638 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118642 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118646 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118650 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118653 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118656 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118660 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118663 2564 flags.go:64] FLAG: --cgroup-root="" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118666 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118669 2564 flags.go:64] FLAG: --client-ca-file="" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118672 2564 flags.go:64] FLAG: --cloud-provider="external" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118674 2564 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118678 2564 flags.go:64] FLAG: --cluster-domain="" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118682 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118685 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118688 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118692 2564 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118696 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 18:31:10.124112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118699 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118702 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118707 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118710 2564 flags.go:64] FLAG: --contention-profiling="false" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118713 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118716 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118719 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118722 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118726 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118729 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118732 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118735 2564 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118738 2564 flags.go:64] FLAG: --enable-server="true" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118741 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118746 2564 flags.go:64] FLAG: --event-burst="100" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118749 2564 flags.go:64] FLAG: --event-qps="50" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118754 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118757 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118760 2564 flags.go:64] FLAG: --eviction-hard="" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118764 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118767 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118769 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118773 2564 flags.go:64] FLAG: --eviction-soft="" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118775 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118778 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 18:31:10.124690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118781 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118784 2564 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118787 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118791 2564 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118794 2564 flags.go:64] FLAG: --feature-gates="" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118797 2564 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118800 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118804 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118807 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118811 2564 flags.go:64] FLAG: --healthz-port="10248" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118814 2564 flags.go:64] FLAG: --help="false" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118817 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118820 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118823 2564 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118826 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118829 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118832 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118835 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118838 2564 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118840 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118843 2564 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118846 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118849 2564 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118853 2564 flags.go:64] FLAG: --kube-reserved="" Sep 10 18:31:10.125364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118856 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118859 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118862 2564 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118864 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118867 2564 flags.go:64] FLAG: --lock-file="" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118870 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118873 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118876 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118882 2564 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118884 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118887 2564 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118890 2564 flags.go:64] FLAG: --logging-format="text" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118893 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118897 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118899 2564 flags.go:64] FLAG: --manifest-url="" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118902 2564 flags.go:64] FLAG: --manifest-url-header="" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118906 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118910 2564 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118914 2564 flags.go:64] FLAG: --max-pods="110" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118917 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118920 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118923 2564 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118926 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118929 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118932 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 18:31:10.125995 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118935 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118942 2564 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118945 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118948 2564 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118951 2564 flags.go:64] FLAG: --pod-cidr="" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118953 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118960 2564 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118963 2564 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118967 2564 flags.go:64] FLAG: --pods-per-core="0" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118970 2564 flags.go:64] FLAG: --port="10250" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118973 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118975 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02f6cd285fa9f883b" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118978 2564 flags.go:64] FLAG: --qos-reserved="" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118981 2564 flags.go:64] FLAG: --read-only-port="10255" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118984 2564 flags.go:64] FLAG: --register-node="true" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118987 2564 flags.go:64] FLAG: --register-with-taints="" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118991 2564 flags.go:64] FLAG: --registry-burst="10" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118994 2564 flags.go:64] FLAG: --registry-qps="5" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.118997 2564 flags.go:64] FLAG: --reserved-cpus="" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119000 2564 flags.go:64] FLAG: --reserved-memory="" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119004 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119007 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119010 2564 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119012 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119015 2564 flags.go:64] FLAG: --runonce="false" Sep 10 18:31:10.126706 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119019 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119022 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119025 2564 flags.go:64] FLAG: --seccomp-default="false" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119028 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119031 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119047 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119051 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119054 2564 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119056 2564 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119059 2564 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119062 2564 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119065 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119068 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119073 2564 flags.go:64] FLAG: --system-cgroups="" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119076 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119081 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119084 2564 flags.go:64] FLAG: --tls-cert-file="" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119087 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119091 2564 flags.go:64] FLAG: --tls-min-version="" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119093 2564 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119096 2564 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119099 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119102 2564 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119105 2564 flags.go:64] FLAG: --v="2" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119108 2564 flags.go:64] FLAG: --version="false" Sep 10 18:31:10.127631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119113 2564 flags.go:64] FLAG: --vmodule="" Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119116 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119120 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119254 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119258 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119261 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119264 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119268 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119272 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119275 2564 feature_gate.go:328] unrecognized feature gate: Example Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119278 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119281 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119284 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119287 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119291 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119295 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119298 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119301 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119304 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 18:31:10.128496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119307 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119312 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119315 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119318 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119321 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119323 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119326 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119329 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119331 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119334 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119336 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119339 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119341 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119344 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119347 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119349 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119352 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119355 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119357 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119360 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 18:31:10.128976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119364 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119366 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119369 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119372 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119374 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119378 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119381 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119384 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119387 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119390 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119392 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119395 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119397 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119401 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119404 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119406 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119409 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119412 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119414 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 18:31:10.129496 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119417 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119419 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119422 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119424 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119427 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119430 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119432 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119435 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119437 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119440 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119442 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119446 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119449 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119452 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119455 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119457 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119460 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119462 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119465 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119467 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 18:31:10.129958 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119470 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119472 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119475 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119478 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119480 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119482 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119486 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119489 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119491 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119494 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119496 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119499 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119501 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119506 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119509 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119512 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119514 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119517 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119519 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 18:31:10.130490 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119522 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 18:31:10.130965 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119524 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 18:31:10.130965 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119527 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 18:31:10.130965 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119530 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 18:31:10.130965 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.119532 2564 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 18:31:10.130965 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.119537 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 18:31:10.130965 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.129235 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 18:31:10.130965 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.129251 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 18:31:10.130965 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.130176 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 18:31:10.130965 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.130187 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 18:31:10.130965 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130246 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 18:31:10.130965 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130251 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 18:31:10.130965 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130255 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 18:31:10.130965 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130258 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 18:31:10.130965 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130261 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 18:31:10.130965 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130263 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130266 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130269 2564 feature_gate.go:328] unrecognized feature gate: Example Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130271 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130274 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130276 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130280 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130285 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130288 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130291 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130294 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130297 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130299 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130302 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130304 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130307 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130313 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130316 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130318 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130321 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 18:31:10.131507 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130324 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130326 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130329 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130332 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130336 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130338 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130341 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130343 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130346 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130349 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130352 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130354 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130357 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130359 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130362 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130365 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130367 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130370 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130373 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130375 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 18:31:10.132005 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130378 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130380 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130383 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130386 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130389 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130391 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130394 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130396 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130399 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130401 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130404 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130407 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130410 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130412 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130415 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130418 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130422 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130425 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130428 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 18:31:10.132518 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130430 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130434 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130436 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130439 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130443 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130446 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130450 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130452 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130455 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130458 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130461 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130464 2564 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130467 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130469 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130472 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130475 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130478 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130481 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130484 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130487 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 18:31:10.132976 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130491 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 18:31:10.133480 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130494 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 18:31:10.133480 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130498 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 18:31:10.133480 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130501 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 18:31:10.133480 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130504 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 18:31:10.133480 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130507 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 18:31:10.133480 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130510 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 18:31:10.133480 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130513 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 18:31:10.133480 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130516 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 18:31:10.133480 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130519 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 18:31:10.133480 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130522 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 18:31:10.133480 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130525 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 18:31:10.133480 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130528 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 18:31:10.133480 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130531 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 18:31:10.133480 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130533 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 18:31:10.133819 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.130538 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 18:31:10.133819 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130667 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 18:31:10.133819 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130673 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 18:31:10.133819 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130676 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 18:31:10.133819 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130679 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 18:31:10.133819 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130682 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 18:31:10.133819 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130685 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 18:31:10.133819 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130688 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 18:31:10.133819 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130691 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 18:31:10.133819 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130694 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 18:31:10.133819 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130696 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 18:31:10.133819 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130699 2564 feature_gate.go:328] unrecognized feature gate: Example Sep 10 18:31:10.133819 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130701 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 18:31:10.133819 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130705 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 18:31:10.133819 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130707 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130710 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130712 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130715 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130717 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130720 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130722 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130725 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130728 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130730 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130733 2564 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130736 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130738 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130741 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130744 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130746 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130749 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130751 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130755 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 18:31:10.134222 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130759 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130762 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130765 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130767 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130770 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130772 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130775 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130777 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130779 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130782 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130784 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130787 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130790 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130793 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130795 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130797 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130800 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130803 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130805 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 18:31:10.134684 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130809 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130813 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130816 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130819 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130822 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130825 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130827 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130830 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130833 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130836 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130839 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130842 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130844 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130847 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130849 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130853 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130856 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130859 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130862 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 18:31:10.135152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130864 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130867 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130869 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130872 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130874 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130877 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130879 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130882 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130885 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130888 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130890 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130893 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130895 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130898 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130901 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130904 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130906 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130909 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130912 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130915 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 18:31:10.135601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130917 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 18:31:10.136099 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130920 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 18:31:10.136099 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130922 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 18:31:10.136099 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130925 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 18:31:10.136099 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130927 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 18:31:10.136099 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130930 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 18:31:10.136099 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130933 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 18:31:10.136099 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130935 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 18:31:10.136099 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:10.130938 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 18:31:10.136099 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.130947 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 18:31:10.136099 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.131093 2564 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 18:31:10.136099 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.134402 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 18:31:10.136099 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.135442 2564 server.go:1019] "Starting client certificate rotation" Sep 10 18:31:10.136099 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.135543 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 18:31:10.136099 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.135574 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 18:31:10.138527 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.138513 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 18:31:10.139667 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.139649 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 18:31:10.149533 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.149513 2564 log.go:25] "Validated CRI v1 runtime API" Sep 10 18:31:10.153620 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.153601 2564 log.go:25] "Validated CRI v1 image API" Sep 10 18:31:10.154545 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.154524 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 18:31:10.158608 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.158589 2564 fs.go:135] Filesystem UUIDs: map[47fad6f3-b8fb-467d-b6c9-19814020cb64:/dev/nvme0n1p3 7481f52b-b98f-4c7d-8e6b-0504c445adbb:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Sep 10 18:31:10.158670 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.158607 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 18:31:10.162212 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.162196 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 18:31:10.164680 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.164568 2564 manager.go:216] Machine: {Timestamp:2026-09-10 18:31:10.162169781 +0000 UTC m=+0.374918885 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102038 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22d943ea0d89a670285eef0a7ac92b SystemUUID:ec22d943-ea0d-89a6-7028-5eef0a7ac92b BootID:4b4d740d-0977-476d-8d49-0cb78f144f57 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9c:22:c3:a1:1f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9c:22:c3:a1:1f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:5c:79:7e:ce:68 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 18:31:10.164680 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.164676 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 18:31:10.164807 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.164758 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 18:31:10.168240 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.168216 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 18:31:10.168406 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.168244 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-87.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 18:31:10.169161 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.169149 2564 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 18:31:10.169217 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.169166 2564 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 18:31:10.169217 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.169178 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 18:31:10.170015 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.170004 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 18:31:10.171852 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.171840 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 18:31:10.176362 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.176348 2564 state_mem.go:36] "Initialized new in-memory state store" Sep 10 18:31:10.176523 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.176511 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 18:31:10.176580 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.176549 2564 kubelet.go:486] "Attempting to sync node with API server" Sep 10 18:31:10.176580 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.176564 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 18:31:10.176666 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.176584 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 18:31:10.176666 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.176600 2564 kubelet.go:398] "Adding apiserver pod source" Sep 10 18:31:10.176666 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.176613 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 18:31:10.177002 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.176988 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 18:31:10.177083 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.177013 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 18:31:10.180766 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.180747 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 18:31:10.183830 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.183815 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 18:31:10.183887 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.183845 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 18:31:10.184080 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.184069 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 18:31:10.184116 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.184085 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 18:31:10.184116 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.184092 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 18:31:10.184116 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.184097 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 18:31:10.184116 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.184105 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 18:31:10.184116 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.184110 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 18:31:10.184646 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.184638 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 18:31:10.184646 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.184645 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 18:31:10.184700 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.184658 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 18:31:10.184700 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.184664 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 18:31:10.184700 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.184671 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 18:31:10.184700 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.184680 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 18:31:10.185495 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.185486 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 18:31:10.185528 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.185497 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 18:31:10.189487 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.189473 2564 server.go:1268] "Started kubelet" Sep 10 18:31:10.189622 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.189553 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 18:31:10.189681 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.189571 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 18:31:10.189681 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.189658 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 18:31:10.189883 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.189858 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-87.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 18:31:10.190009 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.189994 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-87.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 18:31:10.190115 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.190097 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 18:31:10.190422 ip-10-0-138-87 systemd[1]: Started Kubernetes Kubelet. Sep 10 18:31:10.191111 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.191082 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 18:31:10.191707 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.191690 2564 server.go:310] "Adding debug handlers to kubelet server" Sep 10 18:31:10.195161 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.195128 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 18:31:10.197258 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.197240 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 18:31:10.197764 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.196581 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-87.ec2.internal.18d40933d4b387a2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-87.ec2.internal,UID:ip-10-0-138-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-87.ec2.internal,},FirstTimestamp:2026-09-10 18:31:10.189447074 +0000 UTC m=+0.402196177,LastTimestamp:2026-09-10 18:31:10.189447074 +0000 UTC m=+0.402196177,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-87.ec2.internal,}" Sep 10 18:31:10.198061 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.198031 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 18:31:10.198194 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.198176 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:10.199077 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.198936 2564 factory.go:55] Registering systemd factory Sep 10 18:31:10.199141 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.199087 2564 factory.go:223] Registration of the systemd container factory successfully Sep 10 18:31:10.199141 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.198997 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 18:31:10.199215 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.198999 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 18:31:10.199215 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.199152 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 18:31:10.199282 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.199248 2564 reconstruct.go:97] "Volume reconstruction finished" Sep 10 18:31:10.199282 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.199256 2564 reconciler.go:29] "Reconciler: start to sync state" Sep 10 18:31:10.199339 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.199291 2564 factory.go:153] Registering CRI-O factory Sep 10 18:31:10.199339 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.199303 2564 factory.go:223] Registration of the crio container factory successfully Sep 10 18:31:10.199390 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.199348 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 18:31:10.199390 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.199366 2564 factory.go:103] Registering Raw factory Sep 10 18:31:10.199390 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.199378 2564 manager.go:1196] Started watching for new ooms in manager Sep 10 18:31:10.199769 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.199757 2564 manager.go:349] Starting recovery of all containers Sep 10 18:31:10.207010 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.206927 2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-87.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 18:31:10.207166 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.207142 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 18:31:10.213157 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.213140 2564 manager.go:354] Recovery completed Sep 10 18:31:10.217618 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.217605 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 18:31:10.219761 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.219745 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasSufficientMemory" Sep 10 18:31:10.219828 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.219773 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 18:31:10.219828 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.219784 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasSufficientPID" Sep 10 18:31:10.220347 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.220327 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 18:31:10.220347 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.220346 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 18:31:10.220429 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.220365 2564 state_mem.go:36] "Initialized new in-memory state store" Sep 10 18:31:10.220429 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.220337 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 18:31:10.221379 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.221365 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasSufficientMemory" Sep 10 18:31:10.221438 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.221394 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 18:31:10.221438 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.221408 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasSufficientPID" Sep 10 18:31:10.222125 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.222057 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-87.ec2.internal.18d40933d6821376 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-87.ec2.internal,UID:ip-10-0-138-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-87.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-87.ec2.internal,},FirstTimestamp:2026-09-10 18:31:10.219760502 +0000 UTC m=+0.432509604,LastTimestamp:2026-09-10 18:31:10.219760502 +0000 UTC m=+0.432509604,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-87.ec2.internal,}" Sep 10 18:31:10.225656 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.225636 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5mxk6" Sep 10 18:31:10.229447 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.229373 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-87.ec2.internal.18d40933d6825935 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-87.ec2.internal,UID:ip-10-0-138-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-87.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-87.ec2.internal,},FirstTimestamp:2026-09-10 18:31:10.219778357 +0000 UTC m=+0.432527460,LastTimestamp:2026-09-10 18:31:10.219778357 +0000 UTC m=+0.432527460,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-87.ec2.internal,}" Sep 10 18:31:10.234090 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.234071 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5mxk6" Sep 10 18:31:10.241255 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.241191 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-87.ec2.internal.18d40933d6827dbf default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-87.ec2.internal,UID:ip-10-0-138-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-87.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-87.ec2.internal,},FirstTimestamp:2026-09-10 18:31:10.219787711 +0000 UTC m=+0.432536814,LastTimestamp:2026-09-10 18:31:10.219787711 +0000 UTC m=+0.432536814,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-87.ec2.internal,}" Sep 10 18:31:10.251165 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.251106 2564 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-87.ec2.internal.18d40933d6821376\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-87.ec2.internal.18d40933d6821376 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-87.ec2.internal,UID:ip-10-0-138-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-87.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-87.ec2.internal,},FirstTimestamp:2026-09-10 18:31:10.219760502 +0000 UTC m=+0.432509604,LastTimestamp:2026-09-10 18:31:10.221379301 +0000 UTC m=+0.434128408,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-87.ec2.internal,}" Sep 10 18:31:10.369372 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.262118 2564 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-87.ec2.internal.18d40933d6825935\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-87.ec2.internal.18d40933d6825935 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-87.ec2.internal,UID:ip-10-0-138-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-87.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-87.ec2.internal,},FirstTimestamp:2026-09-10 18:31:10.219778357 +0000 UTC m=+0.432527460,LastTimestamp:2026-09-10 18:31:10.2214004 +0000 UTC m=+0.434149505,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-87.ec2.internal,}" Sep 10 18:31:10.369372 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.270315 2564 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-87.ec2.internal.18d40933d6827dbf\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-87.ec2.internal.18d40933d6827dbf default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-87.ec2.internal,UID:ip-10-0-138-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-87.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-87.ec2.internal,},FirstTimestamp:2026-09-10 18:31:10.219787711 +0000 UTC m=+0.432536814,LastTimestamp:2026-09-10 18:31:10.221413646 +0000 UTC m=+0.434162750,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-87.ec2.internal,}" Sep 10 18:31:10.369372 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.298483 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:10.369372 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.368346 2564 policy_none.go:49] "None policy: Start" Sep 10 18:31:10.369372 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.368367 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 18:31:10.369372 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.368382 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 18:31:10.369924 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.369884 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 18:31:10.370605 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.370586 2564 policy_none.go:47] "Start" Sep 10 18:31:10.371468 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.371451 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 18:31:10.371554 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.371511 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 18:31:10.371554 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.371538 2564 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 18:31:10.371652 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.371584 2564 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Sep 10 18:31:10.385266 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.385251 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 18:31:10.398897 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.398878 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:10.404562 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.404547 2564 manager.go:337] "Starting Device Plugin manager" Sep 10 18:31:10.404638 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.404583 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 18:31:10.404638 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.404593 2564 server.go:87] "Starting device plugin registration server" Sep 10 18:31:10.404912 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.404901 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 18:31:10.404950 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.404914 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 18:31:10.405026 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.405008 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 18:31:10.405136 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.405099 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 18:31:10.405136 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.405107 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 18:31:10.405556 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.405538 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 18:31:10.405632 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.405579 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:10.415067 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.415047 2564 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-87.ec2.internal\" not found" node="ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.472271 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.472246 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-87.ec2.internal"] Sep 10 18:31:10.472381 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.472346 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 18:31:10.473245 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.473225 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasSufficientMemory" Sep 10 18:31:10.473334 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.473259 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 18:31:10.473334 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.473275 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasSufficientPID" Sep 10 18:31:10.475617 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.475602 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 18:31:10.475728 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.475715 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.475779 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.475745 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 18:31:10.476305 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.476288 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasSufficientMemory" Sep 10 18:31:10.476398 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.476318 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 18:31:10.476398 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.476332 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasSufficientPID" Sep 10 18:31:10.476398 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.476289 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasSufficientMemory" Sep 10 18:31:10.476398 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.476391 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 18:31:10.476581 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.476403 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasSufficientPID" Sep 10 18:31:10.478579 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.478563 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.478645 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.478594 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 18:31:10.479306 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.479290 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasSufficientMemory" Sep 10 18:31:10.479371 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.479327 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 18:31:10.479371 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.479342 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasSufficientPID" Sep 10 18:31:10.497530 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.497514 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-87.ec2.internal\" not found" node="ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.500230 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.500195 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/40134ba3305c34e71775d79f7ccc18ed-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal\" (UID: \"40134ba3305c34e71775d79f7ccc18ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.500230 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.500225 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40134ba3305c34e71775d79f7ccc18ed-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal\" (UID: \"40134ba3305c34e71775d79f7ccc18ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.500349 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.500250 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7fc4c017da0da160b512c920380689c3-config\") pod \"kube-apiserver-proxy-ip-10-0-138-87.ec2.internal\" (UID: \"7fc4c017da0da160b512c920380689c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.501808 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.501794 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-87.ec2.internal\" not found" node="ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.504991 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.504975 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 18:31:10.505684 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.505666 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasSufficientMemory" Sep 10 18:31:10.505757 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.505695 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 18:31:10.505757 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.505705 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasSufficientPID" Sep 10 18:31:10.505757 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.505726 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.514300 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.514287 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.514352 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.514305 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-87.ec2.internal\": node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:10.543861 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.543837 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:10.601307 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.601279 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40134ba3305c34e71775d79f7ccc18ed-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal\" (UID: \"40134ba3305c34e71775d79f7ccc18ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.601439 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.601316 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7fc4c017da0da160b512c920380689c3-config\") pod \"kube-apiserver-proxy-ip-10-0-138-87.ec2.internal\" (UID: \"7fc4c017da0da160b512c920380689c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.601439 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.601339 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/40134ba3305c34e71775d79f7ccc18ed-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal\" (UID: \"40134ba3305c34e71775d79f7ccc18ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.601439 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.601364 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/40134ba3305c34e71775d79f7ccc18ed-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal\" (UID: \"40134ba3305c34e71775d79f7ccc18ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.601439 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.601363 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40134ba3305c34e71775d79f7ccc18ed-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal\" (UID: \"40134ba3305c34e71775d79f7ccc18ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.601439 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.601389 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7fc4c017da0da160b512c920380689c3-config\") pod \"kube-apiserver-proxy-ip-10-0-138-87.ec2.internal\" (UID: \"7fc4c017da0da160b512c920380689c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.644180 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.644131 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:10.744830 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.744806 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:10.801169 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.801151 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.806751 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:10.806425 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-87.ec2.internal" Sep 10 18:31:10.845624 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.845599 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:10.946236 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:10.946150 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:11.046628 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:11.046605 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:12.204748 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:12.204718 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 18:31:12.205141 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:12.204868 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 18:31:12.217004 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:12.216982 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:12.308342 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:12.308276 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 18:31:12.317513 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:12.317496 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:12.347657 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:12.347628 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 18:31:12.348639 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:12.348622 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasSufficientMemory" Sep 10 18:31:12.348700 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:12.348654 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 18:31:12.348700 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:12.348665 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeHasSufficientPID" Sep 10 18:31:12.418256 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:12.418232 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:12.438424 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:12.438377 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 18:26:11 +0000 UTC" deadline="2028-04-26 19:54:44.034734169 +0000 UTC" Sep 10 18:31:12.438424 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:12.438422 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14257h23m31.596315787s" Sep 10 18:31:12.454516 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:12.454479 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 18:31:12.518544 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:12.518521 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:12.619042 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:12.618973 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:12.642578 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:12.642562 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 18:31:12.720043 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:12.720000 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:12.820716 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:12.820679 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-87.ec2.internal\" not found" Sep 10 18:31:12.831654 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:12.831634 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 18:31:12.868190 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:12.868169 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal" Sep 10 18:31:12.880200 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:12.880156 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 18:31:12.881173 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:12.881160 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-87.ec2.internal" Sep 10 18:31:12.886822 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:12.886808 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 18:31:13.224818 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.224742 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 18:31:13.247179 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.247154 2564 apiserver.go:52] "Watching apiserver" Sep 10 18:31:13.252606 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.252516 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 18:31:13.252937 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.252915 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-7h88j","openshift-image-registry/node-ca-cd2ls","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal","openshift-multus/multus-6jw7x","openshift-multus/network-metrics-daemon-6dxjk","openshift-network-diagnostics/network-check-target-dnfs8","openshift-network-operator/iptables-alerter-6thwj","kube-system/konnectivity-agent-q4chw","kube-system/kube-apiserver-proxy-ip-10-0-138-87.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p","openshift-dns/node-resolver-4lr7x","openshift-multus/multus-additional-cni-plugins-nrlp8","openshift-ovn-kubernetes/ovnkube-node-pb2pr","kube-system/global-pull-secret-syncer-mm2nc"] Sep 10 18:31:13.256078 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.256055 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6thwj" Sep 10 18:31:13.258549 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.258527 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.258935 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.258913 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 18:31:13.259429 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.259124 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 18:31:13.260096 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.259543 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wnxd8\"" Sep 10 18:31:13.260691 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.259386 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 18:31:13.260691 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.260551 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 18:31:13.261363 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.260843 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 18:31:13.261363 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.260979 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 18:31:13.261363 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.261050 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 18:31:13.261363 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.261101 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 18:31:13.261363 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.261126 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5qkpr\"" Sep 10 18:31:13.261363 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.261050 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 18:31:13.261757 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.261383 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:13.261757 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.261463 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:13.263690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.263543 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.266105 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.266001 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-z6k6k\"" Sep 10 18:31:13.266170 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.266155 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 18:31:13.266458 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.266279 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 18:31:13.266458 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.266322 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 18:31:13.266458 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.266379 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 18:31:13.266675 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.266576 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:13.266675 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.266629 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:13.267474 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.267455 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 18:31:13.268685 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.268664 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:13.268786 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.268751 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:13.272070 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.272032 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q4chw" Sep 10 18:31:13.275050 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.274761 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 18:31:13.275050 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.274857 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cxsx9\"" Sep 10 18:31:13.275050 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.274879 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 18:31:13.275698 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.275373 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.277969 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.277529 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 18:31:13.277969 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.277959 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cd2ls" Sep 10 18:31:13.278565 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278544 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs\") pod \"network-metrics-daemon-6dxjk\" (UID: \"99da651c-7b27-49ea-b49a-7f72caf21100\") " pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:13.278682 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278578 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/25dec05c-c81f-4c62-a6e8-1d3506812ff5-kubelet-config\") pod \"global-pull-secret-syncer-mm2nc\" (UID: \"25dec05c-c81f-4c62-a6e8-1d3506812ff5\") " pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:13.278682 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278608 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-systemd-units\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.278682 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278631 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-run-netns\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.278682 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278654 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/019680e6-9104-47fb-b272-9ced99c8bfc8-ovnkube-config\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.278682 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278677 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-cnibin\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278698 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e20510c-c392-4fc1-a566-e2b9179d3e08-cni-binary-copy\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278720 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-var-lib-cni-multus\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278745 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-hostroot\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278748 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-s2hdg\"" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278771 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d0a0a9fe-c357-48a7-ac8d-086772b736d8-iptables-alerter-script\") pod \"iptables-alerter-6thwj\" (UID: \"d0a0a9fe-c357-48a7-ac8d-086772b736d8\") " pod="openshift-network-operator/iptables-alerter-6thwj" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278794 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-log-socket\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278816 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-multus-socket-dir-parent\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278838 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-run-netns\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278860 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3e20510c-c392-4fc1-a566-e2b9179d3e08-multus-daemon-config\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278882 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-etc-kubernetes\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278905 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cclcr\" (UniqueName: \"kubernetes.io/projected/d0a0a9fe-c357-48a7-ac8d-086772b736d8-kube-api-access-cclcr\") pod \"iptables-alerter-6thwj\" (UID: \"d0a0a9fe-c357-48a7-ac8d-086772b736d8\") " pod="openshift-network-operator/iptables-alerter-6thwj" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278927 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/25dec05c-c81f-4c62-a6e8-1d3506812ff5-dbus\") pod \"global-pull-secret-syncer-mm2nc\" (UID: \"25dec05c-c81f-4c62-a6e8-1d3506812ff5\") " pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278948 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/97385966-6ff5-44b8-9a49-cfb01d67fefe-agent-certs\") pod \"konnectivity-agent-q4chw\" (UID: \"97385966-6ff5-44b8-9a49-cfb01d67fefe\") " pod="kube-system/konnectivity-agent-q4chw" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278954 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.278979 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/019680e6-9104-47fb-b272-9ced99c8bfc8-ovn-node-metrics-cert\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279001 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-var-lib-kubelet\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279025 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gr9s\" (UniqueName: \"kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s\") pod \"network-check-target-dnfs8\" (UID: \"4f84d0cb-448c-4000-85cf-68e4e014c8ec\") " pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:13.279084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279048 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-slash\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279071 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-cni-netd\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279121 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnxh2\" (UniqueName: \"kubernetes.io/projected/019680e6-9104-47fb-b272-9ced99c8bfc8-kube-api-access-jnxh2\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279204 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-multus-cni-dir\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279229 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-var-lib-cni-bin\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279257 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-multus-conf-dir\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279281 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4f5n\" (UniqueName: \"kubernetes.io/projected/3e20510c-c392-4fc1-a566-e2b9179d3e08-kube-api-access-c4f5n\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279303 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0a0a9fe-c357-48a7-ac8d-086772b736d8-host-slash\") pod \"iptables-alerter-6thwj\" (UID: \"d0a0a9fe-c357-48a7-ac8d-086772b736d8\") " pod="openshift-network-operator/iptables-alerter-6thwj" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279324 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-run-ovn\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279345 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/019680e6-9104-47fb-b272-9ced99c8bfc8-env-overrides\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279368 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-system-cni-dir\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279389 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-run-systemd\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279421 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-etc-openvswitch\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279444 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-node-log\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279517 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-cni-bin\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279543 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279566 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-run-k8s-cni-cncf-io\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.279889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279588 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-run-multus-certs\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.280635 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279612 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-var-lib-openvswitch\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.280635 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279635 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-run-openvswitch\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.280635 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279658 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-run-ovn-kubernetes\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.280635 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279679 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/019680e6-9104-47fb-b272-9ced99c8bfc8-ovnkube-script-lib\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.280635 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279708 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw8d4\" (UniqueName: \"kubernetes.io/projected/99da651c-7b27-49ea-b49a-7f72caf21100-kube-api-access-lw8d4\") pod \"network-metrics-daemon-6dxjk\" (UID: \"99da651c-7b27-49ea-b49a-7f72caf21100\") " pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:13.280635 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279732 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret\") pod \"global-pull-secret-syncer-mm2nc\" (UID: \"25dec05c-c81f-4c62-a6e8-1d3506812ff5\") " pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:13.280635 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279757 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/97385966-6ff5-44b8-9a49-cfb01d67fefe-konnectivity-ca\") pod \"konnectivity-agent-q4chw\" (UID: \"97385966-6ff5-44b8-9a49-cfb01d67fefe\") " pod="kube-system/konnectivity-agent-q4chw" Sep 10 18:31:13.280635 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.279821 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-kubelet\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.280635 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.280096 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.280635 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.280588 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 18:31:13.280635 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.280635 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sqmnq\"" Sep 10 18:31:13.281126 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.280783 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 18:31:13.281126 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.280855 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 18:31:13.281863 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.281845 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 18:31:13.282172 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.282152 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 18:31:13.282249 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.282207 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 18:31:13.282814 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.282791 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 18:31:13.283127 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.282842 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fz55j\"" Sep 10 18:31:13.283127 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.283095 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4lr7x" Sep 10 18:31:13.285521 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.285268 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.286135 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.286105 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 18:31:13.286305 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.286287 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 18:31:13.286532 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.286468 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6fc2f\"" Sep 10 18:31:13.288161 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.287352 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 18:31:13.288161 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.287717 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 18:31:13.288161 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.287982 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-r9zwv\"" Sep 10 18:31:13.319918 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.319897 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cxbdp" Sep 10 18:31:13.329372 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.329354 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cxbdp" Sep 10 18:31:13.370316 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.370247 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 18:31:13.380632 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.380579 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-kubelet\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.380632 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.380621 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/25dec05c-c81f-4c62-a6e8-1d3506812ff5-kubelet-config\") pod \"global-pull-secret-syncer-mm2nc\" (UID: \"25dec05c-c81f-4c62-a6e8-1d3506812ff5\") " pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:13.380814 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.380654 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-sysconfig\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.380814 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.380673 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-sysctl-conf\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.380814 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.380688 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-etc-selinux\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.380814 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.380709 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmhhb\" (UniqueName: \"kubernetes.io/projected/92d1a4d6-3b6a-4676-b798-8ca176789fb1-kube-api-access-nmhhb\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.380814 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.380725 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-systemd-units\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.380814 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.380745 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e20510c-c392-4fc1-a566-e2b9179d3e08-cni-binary-copy\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.380814 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.380759 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-var-lib-cni-multus\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.380814 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.380772 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-var-lib-kubelet\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.380814 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.380786 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-host\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.380814 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.380799 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.380814 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.380812 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0-tmp-dir\") pod \"node-resolver-4lr7x\" (UID: \"e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0\") " pod="openshift-dns/node-resolver-4lr7x" Sep 10 18:31:13.381341 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.380827 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-multus-socket-dir-parent\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.381341 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.380872 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3e20510c-c392-4fc1-a566-e2b9179d3e08-multus-daemon-config\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.381341 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381036 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-multus-socket-dir-parent\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.381341 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381085 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-var-lib-cni-multus\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.381341 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381116 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-etc-kubernetes\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.381341 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381126 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-systemd-units\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.381341 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381150 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-kubelet\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.381341 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381211 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/25dec05c-c81f-4c62-a6e8-1d3506812ff5-kubelet-config\") pod \"global-pull-secret-syncer-mm2nc\" (UID: \"25dec05c-c81f-4c62-a6e8-1d3506812ff5\") " pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:13.381341 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381299 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-etc-kubernetes\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.381341 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381332 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cclcr\" (UniqueName: \"kubernetes.io/projected/d0a0a9fe-c357-48a7-ac8d-086772b736d8-kube-api-access-cclcr\") pod \"iptables-alerter-6thwj\" (UID: \"d0a0a9fe-c357-48a7-ac8d-086772b736d8\") " pod="openshift-network-operator/iptables-alerter-6thwj" Sep 10 18:31:13.381341 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381349 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/97385966-6ff5-44b8-9a49-cfb01d67fefe-agent-certs\") pod \"konnectivity-agent-q4chw\" (UID: \"97385966-6ff5-44b8-9a49-cfb01d67fefe\") " pod="kube-system/konnectivity-agent-q4chw" Sep 10 18:31:13.381890 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381576 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/019680e6-9104-47fb-b272-9ced99c8bfc8-ovn-node-metrics-cert\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.381890 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381673 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-slash\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.381890 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381733 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnxh2\" (UniqueName: \"kubernetes.io/projected/019680e6-9104-47fb-b272-9ced99c8bfc8-kube-api-access-jnxh2\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.381890 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381755 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-multus-cni-dir\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.381890 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381838 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-var-lib-cni-bin\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.381890 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381865 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-multus-conf-dir\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.381890 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381883 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0a0a9fe-c357-48a7-ac8d-086772b736d8-host-slash\") pod \"iptables-alerter-6thwj\" (UID: \"d0a0a9fe-c357-48a7-ac8d-086772b736d8\") " pod="openshift-network-operator/iptables-alerter-6thwj" Sep 10 18:31:13.382210 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381897 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/019680e6-9104-47fb-b272-9ced99c8bfc8-env-overrides\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.382210 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.381933 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0b66f99-0213-4038-9319-be325dc2afad-tmp\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.382210 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.382053 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/92d1a4d6-3b6a-4676-b798-8ca176789fb1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.382210 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.382099 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/92d1a4d6-3b6a-4676-b798-8ca176789fb1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.382210 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.382131 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-run-systemd\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.382210 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.382184 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-cni-bin\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.382510 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.382212 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-run-k8s-cni-cncf-io\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.382510 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.382251 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-run-multus-certs\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.382510 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.382275 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw8d4\" (UniqueName: \"kubernetes.io/projected/99da651c-7b27-49ea-b49a-7f72caf21100-kube-api-access-lw8d4\") pod \"network-metrics-daemon-6dxjk\" (UID: \"99da651c-7b27-49ea-b49a-7f72caf21100\") " pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:13.382510 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.382301 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/97385966-6ff5-44b8-9a49-cfb01d67fefe-konnectivity-ca\") pod \"konnectivity-agent-q4chw\" (UID: \"97385966-6ff5-44b8-9a49-cfb01d67fefe\") " pod="kube-system/konnectivity-agent-q4chw" Sep 10 18:31:13.382510 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.382336 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-sys\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.382510 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.382467 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-var-lib-openvswitch\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.382510 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.382499 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-run-ovn-kubernetes\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.382838 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.382550 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e20510c-c392-4fc1-a566-e2b9179d3e08-cni-binary-copy\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.382838 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.382566 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret\") pod \"global-pull-secret-syncer-mm2nc\" (UID: \"25dec05c-c81f-4c62-a6e8-1d3506812ff5\") " pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:13.382838 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.382602 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 18:31:13.382838 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.382664 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 18:31:13.382838 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.382660 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-systemd\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.382838 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.382700 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b0b66f99-0213-4038-9319-be325dc2afad-etc-tuned\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.383175 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.383139 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret podName:25dec05c-c81f-4c62-a6e8-1d3506812ff5 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:13.883103161 +0000 UTC m=+3.026406400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret") pod "global-pull-secret-syncer-mm2nc" (UID: "25dec05c-c81f-4c62-a6e8-1d3506812ff5") : object "kube-system"/"original-pull-secret" not registered Sep 10 18:31:13.384198 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.384131 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/019680e6-9104-47fb-b272-9ced99c8bfc8-env-overrides\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.384584 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.384557 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3e20510c-c392-4fc1-a566-e2b9179d3e08-multus-daemon-config\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.386012 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.384923 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-slash\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.386012 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.385007 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-multus-cni-dir\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.386012 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.385241 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-run-ovn-kubernetes\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.386012 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.385330 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-var-lib-openvswitch\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.386012 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.385382 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-var-lib-cni-bin\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.386012 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.385543 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-cni-bin\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.386012 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.385581 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr27c\" (UniqueName: \"kubernetes.io/projected/b0b66f99-0213-4038-9319-be325dc2afad-kube-api-access-cr27c\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.386012 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.385983 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-multus-conf-dir\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.386389 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.386022 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/96332d76-e827-4612-a6b5-bd9d31fe347e-serviceca\") pod \"node-ca-cd2ls\" (UID: \"96332d76-e827-4612-a6b5-bd9d31fe347e\") " pod="openshift-image-registry/node-ca-cd2ls" Sep 10 18:31:13.386389 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.386071 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-run-systemd\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.386389 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.386111 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-run-k8s-cni-cncf-io\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.386389 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.386186 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0a0a9fe-c357-48a7-ac8d-086772b736d8-host-slash\") pod \"iptables-alerter-6thwj\" (UID: \"d0a0a9fe-c357-48a7-ac8d-086772b736d8\") " pod="openshift-network-operator/iptables-alerter-6thwj" Sep 10 18:31:13.386389 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.386231 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-run-multus-certs\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.386389 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.386291 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92d1a4d6-3b6a-4676-b798-8ca176789fb1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.386693 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.386402 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs\") pod \"network-metrics-daemon-6dxjk\" (UID: \"99da651c-7b27-49ea-b49a-7f72caf21100\") " pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:13.386693 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.386555 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ssxl\" (UniqueName: \"kubernetes.io/projected/96332d76-e827-4612-a6b5-bd9d31fe347e-kube-api-access-8ssxl\") pod \"node-ca-cd2ls\" (UID: \"96332d76-e827-4612-a6b5-bd9d31fe347e\") " pod="openshift-image-registry/node-ca-cd2ls" Sep 10 18:31:13.386693 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.386605 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-registration-dir\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.386693 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.386670 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-device-dir\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.386888 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.386707 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-run-netns\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.386888 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.386733 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/019680e6-9104-47fb-b272-9ced99c8bfc8-ovnkube-config\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.386888 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.386765 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-cnibin\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.387037 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.386911 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-hostroot\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.387037 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.386942 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d0a0a9fe-c357-48a7-ac8d-086772b736d8-iptables-alerter-script\") pod \"iptables-alerter-6thwj\" (UID: \"d0a0a9fe-c357-48a7-ac8d-086772b736d8\") " pod="openshift-network-operator/iptables-alerter-6thwj" Sep 10 18:31:13.387037 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.386972 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/25dec05c-c81f-4c62-a6e8-1d3506812ff5-dbus\") pod \"global-pull-secret-syncer-mm2nc\" (UID: \"25dec05c-c81f-4c62-a6e8-1d3506812ff5\") " pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:13.387037 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.387004 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-socket-dir\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.387194 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.387042 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87bdd\" (UniqueName: \"kubernetes.io/projected/e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0-kube-api-access-87bdd\") pod \"node-resolver-4lr7x\" (UID: \"e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0\") " pod="openshift-dns/node-resolver-4lr7x" Sep 10 18:31:13.387194 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.387061 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/97385966-6ff5-44b8-9a49-cfb01d67fefe-konnectivity-ca\") pod \"konnectivity-agent-q4chw\" (UID: \"97385966-6ff5-44b8-9a49-cfb01d67fefe\") " pod="kube-system/konnectivity-agent-q4chw" Sep 10 18:31:13.387194 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.387084 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-log-socket\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.387194 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.387124 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-run-netns\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.387194 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.387173 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-kubernetes\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.387641 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.387619 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-sysctl-d\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.387781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.387657 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96332d76-e827-4612-a6b5-bd9d31fe347e-host\") pod \"node-ca-cd2ls\" (UID: \"96332d76-e827-4612-a6b5-bd9d31fe347e\") " pod="openshift-image-registry/node-ca-cd2ls" Sep 10 18:31:13.387781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.387689 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92d1a4d6-3b6a-4676-b798-8ca176789fb1-cnibin\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.387781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.387727 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-var-lib-kubelet\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.387929 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.387788 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gr9s\" (UniqueName: \"kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s\") pod \"network-check-target-dnfs8\" (UID: \"4f84d0cb-448c-4000-85cf-68e4e014c8ec\") " pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:13.387929 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.387815 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-sys-fs\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.387929 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.387857 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-cni-netd\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.387929 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.387906 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4f5n\" (UniqueName: \"kubernetes.io/projected/3e20510c-c392-4fc1-a566-e2b9179d3e08-kube-api-access-c4f5n\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.388120 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.387998 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-modprobe-d\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.388120 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.388056 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln98d\" (UniqueName: \"kubernetes.io/projected/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-kube-api-access-ln98d\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.388120 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.388084 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92d1a4d6-3b6a-4676-b798-8ca176789fb1-cni-binary-copy\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.388260 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.388143 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-run-ovn\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.388260 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.388177 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-system-cni-dir\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.388260 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.388218 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92d1a4d6-3b6a-4676-b798-8ca176789fb1-system-cni-dir\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.388260 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.388251 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-etc-openvswitch\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.389080 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.388903 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-node-log\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.389166 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389092 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-cnibin\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.389166 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389043 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-etc-openvswitch\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.389166 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389109 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.389166 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389159 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0-hosts-file\") pod \"node-resolver-4lr7x\" (UID: \"e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0\") " pod="openshift-dns/node-resolver-4lr7x" Sep 10 18:31:13.390014 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.389190 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 18:31:13.390014 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389193 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-run-openvswitch\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.390014 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389226 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/019680e6-9104-47fb-b272-9ced99c8bfc8-ovnkube-script-lib\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.390014 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.389248 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs podName:99da651c-7b27-49ea-b49a-7f72caf21100 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:13.889229191 +0000 UTC m=+3.032532429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs") pod "network-metrics-daemon-6dxjk" (UID: "99da651c-7b27-49ea-b49a-7f72caf21100") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 18:31:13.390014 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389297 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-run\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.390014 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389307 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-run-netns\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.390014 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389326 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-lib-modules\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.390014 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389355 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/25dec05c-c81f-4c62-a6e8-1d3506812ff5-dbus\") pod \"global-pull-secret-syncer-mm2nc\" (UID: \"25dec05c-c81f-4c62-a6e8-1d3506812ff5\") " pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:13.390014 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389423 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-hostroot\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.390014 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.388977 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-run-ovn\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.390014 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389578 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-cni-netd\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.390014 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389653 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-log-socket\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.390014 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389693 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-run-netns\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.390014 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389846 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-host-var-lib-kubelet\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.390014 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389955 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d0a0a9fe-c357-48a7-ac8d-086772b736d8-iptables-alerter-script\") pod \"iptables-alerter-6thwj\" (UID: \"d0a0a9fe-c357-48a7-ac8d-086772b736d8\") " pod="openshift-network-operator/iptables-alerter-6thwj" Sep 10 18:31:13.390014 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.389993 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/019680e6-9104-47fb-b272-9ced99c8bfc8-ovnkube-config\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.391871 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.390080 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-run-openvswitch\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.391871 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.390125 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-node-log\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.391871 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.390171 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/019680e6-9104-47fb-b272-9ced99c8bfc8-ovn-node-metrics-cert\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.391871 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.390312 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e20510c-c392-4fc1-a566-e2b9179d3e08-system-cni-dir\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.391871 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.390365 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/019680e6-9104-47fb-b272-9ced99c8bfc8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.391871 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.390636 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/019680e6-9104-47fb-b272-9ced99c8bfc8-ovnkube-script-lib\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.391871 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.391183 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/97385966-6ff5-44b8-9a49-cfb01d67fefe-agent-certs\") pod \"konnectivity-agent-q4chw\" (UID: \"97385966-6ff5-44b8-9a49-cfb01d67fefe\") " pod="kube-system/konnectivity-agent-q4chw" Sep 10 18:31:13.393624 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.393601 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cclcr\" (UniqueName: \"kubernetes.io/projected/d0a0a9fe-c357-48a7-ac8d-086772b736d8-kube-api-access-cclcr\") pod \"iptables-alerter-6thwj\" (UID: \"d0a0a9fe-c357-48a7-ac8d-086772b736d8\") " pod="openshift-network-operator/iptables-alerter-6thwj" Sep 10 18:31:13.393929 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.393881 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw8d4\" (UniqueName: \"kubernetes.io/projected/99da651c-7b27-49ea-b49a-7f72caf21100-kube-api-access-lw8d4\") pod \"network-metrics-daemon-6dxjk\" (UID: \"99da651c-7b27-49ea-b49a-7f72caf21100\") " pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:13.394831 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.394678 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnxh2\" (UniqueName: \"kubernetes.io/projected/019680e6-9104-47fb-b272-9ced99c8bfc8-kube-api-access-jnxh2\") pod \"ovnkube-node-pb2pr\" (UID: \"019680e6-9104-47fb-b272-9ced99c8bfc8\") " pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.399169 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.399147 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 18:31:13.399265 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.399174 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 18:31:13.399265 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.399187 2564 projected.go:196] Error preparing data for projected volume kube-api-access-8gr9s for pod openshift-network-diagnostics/network-check-target-dnfs8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 18:31:13.399265 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.399261 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s podName:4f84d0cb-448c-4000-85cf-68e4e014c8ec nodeName:}" failed. No retries permitted until 2026-09-10 18:31:13.899243638 +0000 UTC m=+3.042546863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8gr9s" (UniqueName: "kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s") pod "network-check-target-dnfs8" (UID: "4f84d0cb-448c-4000-85cf-68e4e014c8ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 18:31:13.403524 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.403503 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4f5n\" (UniqueName: \"kubernetes.io/projected/3e20510c-c392-4fc1-a566-e2b9179d3e08-kube-api-access-c4f5n\") pod \"multus-6jw7x\" (UID: \"3e20510c-c392-4fc1-a566-e2b9179d3e08\") " pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.447107 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.447069 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal" event={"ID":"40134ba3305c34e71775d79f7ccc18ed","Type":"ContainerStarted","Data":"792820a240ec3dd367077ae4dd5b198970bd9f30ba969ddd17c91eb72e43de79"} Sep 10 18:31:13.448337 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.448312 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-87.ec2.internal" event={"ID":"7fc4c017da0da160b512c920380689c3","Type":"ContainerStarted","Data":"6d2b7b33866b63cd5b7aa9ea838a85fe02df12c08a0d73be4963c18e26fd9b6e"} Sep 10 18:31:13.489757 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489694 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0-hosts-file\") pod \"node-resolver-4lr7x\" (UID: \"e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0\") " pod="openshift-dns/node-resolver-4lr7x" Sep 10 18:31:13.489757 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489736 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-run\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.489915 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489760 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-lib-modules\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.489915 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489783 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-sysconfig\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.489915 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489806 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-sysctl-conf\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.489915 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489825 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-run\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.489915 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489825 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0-hosts-file\") pod \"node-resolver-4lr7x\" (UID: \"e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0\") " pod="openshift-dns/node-resolver-4lr7x" Sep 10 18:31:13.489915 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489830 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-etc-selinux\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.489915 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489870 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-sysconfig\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.489915 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489895 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmhhb\" (UniqueName: \"kubernetes.io/projected/92d1a4d6-3b6a-4676-b798-8ca176789fb1-kube-api-access-nmhhb\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489929 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-var-lib-kubelet\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489938 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-etc-selinux\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489950 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-host\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489960 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-lib-modules\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489982 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489996 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-host\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.489980 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-sysctl-conf\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490013 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0-tmp-dir\") pod \"node-resolver-4lr7x\" (UID: \"e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0\") " pod="openshift-dns/node-resolver-4lr7x" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490047 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0b66f99-0213-4038-9319-be325dc2afad-tmp\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490048 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490014 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-var-lib-kubelet\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490074 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/92d1a4d6-3b6a-4676-b798-8ca176789fb1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490097 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/92d1a4d6-3b6a-4676-b798-8ca176789fb1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490122 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-sys\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490159 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-systemd\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490182 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b0b66f99-0213-4038-9319-be325dc2afad-etc-tuned\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.490225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490204 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr27c\" (UniqueName: \"kubernetes.io/projected/b0b66f99-0213-4038-9319-be325dc2afad-kube-api-access-cr27c\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490227 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/96332d76-e827-4612-a6b5-bd9d31fe347e-serviceca\") pod \"node-ca-cd2ls\" (UID: \"96332d76-e827-4612-a6b5-bd9d31fe347e\") " pod="openshift-image-registry/node-ca-cd2ls" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490251 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-sys\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490307 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92d1a4d6-3b6a-4676-b798-8ca176789fb1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490315 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0-tmp-dir\") pod \"node-resolver-4lr7x\" (UID: \"e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0\") " pod="openshift-dns/node-resolver-4lr7x" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490349 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ssxl\" (UniqueName: \"kubernetes.io/projected/96332d76-e827-4612-a6b5-bd9d31fe347e-kube-api-access-8ssxl\") pod \"node-ca-cd2ls\" (UID: \"96332d76-e827-4612-a6b5-bd9d31fe347e\") " pod="openshift-image-registry/node-ca-cd2ls" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490372 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-registration-dir\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490408 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-device-dir\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490435 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-socket-dir\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490457 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87bdd\" (UniqueName: \"kubernetes.io/projected/e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0-kube-api-access-87bdd\") pod \"node-resolver-4lr7x\" (UID: \"e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0\") " pod="openshift-dns/node-resolver-4lr7x" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490498 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-kubernetes\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490520 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-sysctl-d\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490543 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96332d76-e827-4612-a6b5-bd9d31fe347e-host\") pod \"node-ca-cd2ls\" (UID: \"96332d76-e827-4612-a6b5-bd9d31fe347e\") " pod="openshift-image-registry/node-ca-cd2ls" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490564 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92d1a4d6-3b6a-4676-b798-8ca176789fb1-cnibin\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490603 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-sys-fs\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490620 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-systemd\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490646 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-modprobe-d\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.490925 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490671 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln98d\" (UniqueName: \"kubernetes.io/projected/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-kube-api-access-ln98d\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490696 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92d1a4d6-3b6a-4676-b798-8ca176789fb1-cni-binary-copy\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490714 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/92d1a4d6-3b6a-4676-b798-8ca176789fb1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490733 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/92d1a4d6-3b6a-4676-b798-8ca176789fb1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490723 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92d1a4d6-3b6a-4676-b798-8ca176789fb1-system-cni-dir\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490732 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-socket-dir\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490766 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92d1a4d6-3b6a-4676-b798-8ca176789fb1-system-cni-dir\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490827 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-device-dir\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490844 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-registration-dir\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490864 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96332d76-e827-4612-a6b5-bd9d31fe347e-host\") pod \"node-ca-cd2ls\" (UID: \"96332d76-e827-4612-a6b5-bd9d31fe347e\") " pod="openshift-image-registry/node-ca-cd2ls" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.490924 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-modprobe-d\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.491024 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92d1a4d6-3b6a-4676-b798-8ca176789fb1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.491116 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-kubernetes\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.491168 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-sys-fs\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.491204 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/96332d76-e827-4612-a6b5-bd9d31fe347e-serviceca\") pod \"node-ca-cd2ls\" (UID: \"96332d76-e827-4612-a6b5-bd9d31fe347e\") " pod="openshift-image-registry/node-ca-cd2ls" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.491212 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92d1a4d6-3b6a-4676-b798-8ca176789fb1-cnibin\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.491245 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b0b66f99-0213-4038-9319-be325dc2afad-etc-sysctl-d\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.491749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.491399 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92d1a4d6-3b6a-4676-b798-8ca176789fb1-cni-binary-copy\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.493080 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.493059 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0b66f99-0213-4038-9319-be325dc2afad-tmp\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.493164 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.493115 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b0b66f99-0213-4038-9319-be325dc2afad-etc-tuned\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.499015 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.498965 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr27c\" (UniqueName: \"kubernetes.io/projected/b0b66f99-0213-4038-9319-be325dc2afad-kube-api-access-cr27c\") pod \"tuned-7h88j\" (UID: \"b0b66f99-0213-4038-9319-be325dc2afad\") " pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.499219 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.499199 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmhhb\" (UniqueName: \"kubernetes.io/projected/92d1a4d6-3b6a-4676-b798-8ca176789fb1-kube-api-access-nmhhb\") pod \"multus-additional-cni-plugins-nrlp8\" (UID: \"92d1a4d6-3b6a-4676-b798-8ca176789fb1\") " pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.499552 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.499510 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln98d\" (UniqueName: \"kubernetes.io/projected/f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4-kube-api-access-ln98d\") pod \"aws-ebs-csi-driver-node-mvn4p\" (UID: \"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.499660 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.499568 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87bdd\" (UniqueName: \"kubernetes.io/projected/e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0-kube-api-access-87bdd\") pod \"node-resolver-4lr7x\" (UID: \"e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0\") " pod="openshift-dns/node-resolver-4lr7x" Sep 10 18:31:13.501217 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.501196 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ssxl\" (UniqueName: \"kubernetes.io/projected/96332d76-e827-4612-a6b5-bd9d31fe347e-kube-api-access-8ssxl\") pod \"node-ca-cd2ls\" (UID: \"96332d76-e827-4612-a6b5-bd9d31fe347e\") " pod="openshift-image-registry/node-ca-cd2ls" Sep 10 18:31:13.572422 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.572393 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6thwj" Sep 10 18:31:13.579902 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.579882 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:13.584660 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:13.584628 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0a0a9fe_c357_48a7_ac8d_086772b736d8.slice/crio-83ad7f4955f4d6e0bb879bdd6629a3c4bd6164330477dc8cbbc269bfe95abf32 WatchSource:0}: Error finding container 83ad7f4955f4d6e0bb879bdd6629a3c4bd6164330477dc8cbbc269bfe95abf32: Status 404 returned error can't find the container with id 83ad7f4955f4d6e0bb879bdd6629a3c4bd6164330477dc8cbbc269bfe95abf32 Sep 10 18:31:13.590601 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:13.590576 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod019680e6_9104_47fb_b272_9ced99c8bfc8.slice/crio-ccd46cb88bbd510c9cf9d3df0f845205b9a3344b93fbec3b2931d6bdf29e3c2f WatchSource:0}: Error finding container ccd46cb88bbd510c9cf9d3df0f845205b9a3344b93fbec3b2931d6bdf29e3c2f: Status 404 returned error can't find the container with id ccd46cb88bbd510c9cf9d3df0f845205b9a3344b93fbec3b2931d6bdf29e3c2f Sep 10 18:31:13.591335 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.591268 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6jw7x" Sep 10 18:31:13.600311 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.600289 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q4chw" Sep 10 18:31:13.601719 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:13.601690 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e20510c_c392_4fc1_a566_e2b9179d3e08.slice/crio-8b2f557cfe72b8688aaa1b24ccbfed59f874ea9f04218d8cc9e83766ebd851e9 WatchSource:0}: Error finding container 8b2f557cfe72b8688aaa1b24ccbfed59f874ea9f04218d8cc9e83766ebd851e9: Status 404 returned error can't find the container with id 8b2f557cfe72b8688aaa1b24ccbfed59f874ea9f04218d8cc9e83766ebd851e9 Sep 10 18:31:13.604256 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.604105 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7h88j" Sep 10 18:31:13.612159 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.612100 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cd2ls" Sep 10 18:31:13.616006 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:13.615972 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97385966_6ff5_44b8_9a49_cfb01d67fefe.slice/crio-0133e9c5bd29af6d289ea503d69dda2b9b67884da678cf3a1ec105a9f1f0e681 WatchSource:0}: Error finding container 0133e9c5bd29af6d289ea503d69dda2b9b67884da678cf3a1ec105a9f1f0e681: Status 404 returned error can't find the container with id 0133e9c5bd29af6d289ea503d69dda2b9b67884da678cf3a1ec105a9f1f0e681 Sep 10 18:31:13.620125 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.620102 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" Sep 10 18:31:13.622170 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:13.622146 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0b66f99_0213_4038_9319_be325dc2afad.slice/crio-48ff848141478b7d05c068057f5c641d22a577f7ef99e1c5e5e4144ac00a0873 WatchSource:0}: Error finding container 48ff848141478b7d05c068057f5c641d22a577f7ef99e1c5e5e4144ac00a0873: Status 404 returned error can't find the container with id 48ff848141478b7d05c068057f5c641d22a577f7ef99e1c5e5e4144ac00a0873 Sep 10 18:31:13.627972 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.627951 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4lr7x" Sep 10 18:31:13.634331 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.634188 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nrlp8" Sep 10 18:31:13.640305 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:13.640279 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4bb2ec5_efc3_4db5_a9e9_d0e7ba7e72a0.slice/crio-1d26c81430d4391bcbc7299d266c37e813db75fa04d6a4c8b5ed4d10cc7c7294 WatchSource:0}: Error finding container 1d26c81430d4391bcbc7299d266c37e813db75fa04d6a4c8b5ed4d10cc7c7294: Status 404 returned error can't find the container with id 1d26c81430d4391bcbc7299d266c37e813db75fa04d6a4c8b5ed4d10cc7c7294 Sep 10 18:31:13.649635 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:13.649402 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92d1a4d6_3b6a_4676_b798_8ca176789fb1.slice/crio-c8126ea4242e4e20d95a7818ab79af335d30b9788965c81e0459475a3ca35e98 WatchSource:0}: Error finding container c8126ea4242e4e20d95a7818ab79af335d30b9788965c81e0459475a3ca35e98: Status 404 returned error can't find the container with id c8126ea4242e4e20d95a7818ab79af335d30b9788965c81e0459475a3ca35e98 Sep 10 18:31:13.893625 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.893531 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret\") pod \"global-pull-secret-syncer-mm2nc\" (UID: \"25dec05c-c81f-4c62-a6e8-1d3506812ff5\") " pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:13.893625 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.893580 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs\") pod \"network-metrics-daemon-6dxjk\" (UID: \"99da651c-7b27-49ea-b49a-7f72caf21100\") " pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:13.893849 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.893715 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 18:31:13.893849 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.893750 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 18:31:13.893849 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.893779 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs podName:99da651c-7b27-49ea-b49a-7f72caf21100 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:14.893758563 +0000 UTC m=+4.037061798 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs") pod "network-metrics-daemon-6dxjk" (UID: "99da651c-7b27-49ea-b49a-7f72caf21100") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 18:31:13.894012 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.893864 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret podName:25dec05c-c81f-4c62-a6e8-1d3506812ff5 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:14.893843967 +0000 UTC m=+4.037147205 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret") pod "global-pull-secret-syncer-mm2nc" (UID: "25dec05c-c81f-4c62-a6e8-1d3506812ff5") : object "kube-system"/"original-pull-secret" not registered Sep 10 18:31:13.995152 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:13.995119 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gr9s\" (UniqueName: \"kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s\") pod \"network-check-target-dnfs8\" (UID: \"4f84d0cb-448c-4000-85cf-68e4e014c8ec\") " pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:13.995336 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.995259 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 18:31:13.995336 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.995275 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 18:31:13.995336 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.995284 2564 projected.go:196] Error preparing data for projected volume kube-api-access-8gr9s for pod openshift-network-diagnostics/network-check-target-dnfs8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 18:31:13.995478 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:13.995347 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s podName:4f84d0cb-448c-4000-85cf-68e4e014c8ec nodeName:}" failed. No retries permitted until 2026-09-10 18:31:14.995322691 +0000 UTC m=+4.138625912 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8gr9s" (UniqueName: "kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s") pod "network-check-target-dnfs8" (UID: "4f84d0cb-448c-4000-85cf-68e4e014c8ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 18:31:14.333583 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:14.333416 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 18:26:13 +0000 UTC" deadline="2028-05-03 19:53:12.056324554 +0000 UTC" Sep 10 18:31:14.333583 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:14.333454 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14425h21m57.722874415s" Sep 10 18:31:14.451451 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:14.451415 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nrlp8" event={"ID":"92d1a4d6-3b6a-4676-b798-8ca176789fb1","Type":"ContainerStarted","Data":"c8126ea4242e4e20d95a7818ab79af335d30b9788965c81e0459475a3ca35e98"} Sep 10 18:31:14.452731 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:14.452703 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4lr7x" event={"ID":"e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0","Type":"ContainerStarted","Data":"1d26c81430d4391bcbc7299d266c37e813db75fa04d6a4c8b5ed4d10cc7c7294"} Sep 10 18:31:14.454051 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:14.453996 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" event={"ID":"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4","Type":"ContainerStarted","Data":"ae243a63fed553e4efd2299932143ce582bc489cbf0fc3c61f22b75920f4cace"} Sep 10 18:31:14.455269 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:14.455242 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cd2ls" event={"ID":"96332d76-e827-4612-a6b5-bd9d31fe347e","Type":"ContainerStarted","Data":"12ba3666142a55ee317f31c88d94329add9e35d727874f6fbbb7767c269d3593"} Sep 10 18:31:14.456534 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:14.456510 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q4chw" event={"ID":"97385966-6ff5-44b8-9a49-cfb01d67fefe","Type":"ContainerStarted","Data":"0133e9c5bd29af6d289ea503d69dda2b9b67884da678cf3a1ec105a9f1f0e681"} Sep 10 18:31:14.457790 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:14.457766 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6jw7x" event={"ID":"3e20510c-c392-4fc1-a566-e2b9179d3e08","Type":"ContainerStarted","Data":"8b2f557cfe72b8688aaa1b24ccbfed59f874ea9f04218d8cc9e83766ebd851e9"} Sep 10 18:31:14.458889 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:14.458865 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7h88j" event={"ID":"b0b66f99-0213-4038-9319-be325dc2afad","Type":"ContainerStarted","Data":"48ff848141478b7d05c068057f5c641d22a577f7ef99e1c5e5e4144ac00a0873"} Sep 10 18:31:14.460167 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:14.460146 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" event={"ID":"019680e6-9104-47fb-b272-9ced99c8bfc8","Type":"ContainerStarted","Data":"ccd46cb88bbd510c9cf9d3df0f845205b9a3344b93fbec3b2931d6bdf29e3c2f"} Sep 10 18:31:14.461230 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:14.461212 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6thwj" event={"ID":"d0a0a9fe-c357-48a7-ac8d-086772b736d8","Type":"ContainerStarted","Data":"83ad7f4955f4d6e0bb879bdd6629a3c4bd6164330477dc8cbbc269bfe95abf32"} Sep 10 18:31:14.903661 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:14.903603 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret\") pod \"global-pull-secret-syncer-mm2nc\" (UID: \"25dec05c-c81f-4c62-a6e8-1d3506812ff5\") " pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:14.903847 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:14.903675 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs\") pod \"network-metrics-daemon-6dxjk\" (UID: \"99da651c-7b27-49ea-b49a-7f72caf21100\") " pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:14.915982 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:14.915955 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 18:31:14.917225 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:14.917208 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs podName:99da651c-7b27-49ea-b49a-7f72caf21100 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:16.917185028 +0000 UTC m=+6.060488252 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs") pod "network-metrics-daemon-6dxjk" (UID: "99da651c-7b27-49ea-b49a-7f72caf21100") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 18:31:14.917370 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:14.915959 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 18:31:14.917469 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:14.917461 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret podName:25dec05c-c81f-4c62-a6e8-1d3506812ff5 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:16.91744845 +0000 UTC m=+6.060751690 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret") pod "global-pull-secret-syncer-mm2nc" (UID: "25dec05c-c81f-4c62-a6e8-1d3506812ff5") : object "kube-system"/"original-pull-secret" not registered Sep 10 18:31:15.005068 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:15.004393 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gr9s\" (UniqueName: \"kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s\") pod \"network-check-target-dnfs8\" (UID: \"4f84d0cb-448c-4000-85cf-68e4e014c8ec\") " pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:15.005068 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:15.004570 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 18:31:15.005068 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:15.004592 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 18:31:15.005068 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:15.004607 2564 projected.go:196] Error preparing data for projected volume kube-api-access-8gr9s for pod openshift-network-diagnostics/network-check-target-dnfs8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 18:31:15.005068 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:15.004666 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s podName:4f84d0cb-448c-4000-85cf-68e4e014c8ec nodeName:}" failed. No retries permitted until 2026-09-10 18:31:17.004647033 +0000 UTC m=+6.147950272 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8gr9s" (UniqueName: "kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s") pod "network-check-target-dnfs8" (UID: "4f84d0cb-448c-4000-85cf-68e4e014c8ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 18:31:15.334677 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:15.334576 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 18:26:13 +0000 UTC" deadline="2028-02-04 01:12:42.425775979 +0000 UTC" Sep 10 18:31:15.334677 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:15.334636 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12270h41m27.091144175s" Sep 10 18:31:15.444247 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:15.444031 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:15.444247 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:15.444163 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:15.444456 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:15.444255 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:15.444456 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:15.444339 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:15.444456 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:15.444402 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:15.444628 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:15.444528 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:15.473668 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:15.472727 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-87.ec2.internal" event={"ID":"7fc4c017da0da160b512c920380689c3","Type":"ContainerStarted","Data":"38a4f8f7d8a215a63eeca677963916437f5655de02fe0b2f8450d556e1b45acc"} Sep 10 18:31:16.519177 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:16.519136 2564 generic.go:356] "Generic (PLEG): container finished" podID="40134ba3305c34e71775d79f7ccc18ed" containerID="1fca9596f5f933861448067c1082c304aa102729e31b23c1dfc37c475d2d5beb" exitCode=0 Sep 10 18:31:16.519764 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:16.519212 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal" event={"ID":"40134ba3305c34e71775d79f7ccc18ed","Type":"ContainerDied","Data":"1fca9596f5f933861448067c1082c304aa102729e31b23c1dfc37c475d2d5beb"} Sep 10 18:31:16.535938 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:16.535539 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-87.ec2.internal" podStartSLOduration=4.535518957 podStartE2EDuration="4.535518957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:31:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:31:15.488033192 +0000 UTC m=+4.631336416" watchObservedRunningTime="2026-09-10 18:31:16.535518957 +0000 UTC m=+5.678822203" Sep 10 18:31:16.918580 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:16.918473 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret\") pod \"global-pull-secret-syncer-mm2nc\" (UID: \"25dec05c-c81f-4c62-a6e8-1d3506812ff5\") " pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:16.918580 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:16.918547 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs\") pod \"network-metrics-daemon-6dxjk\" (UID: \"99da651c-7b27-49ea-b49a-7f72caf21100\") " pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:16.918787 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:16.918697 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 18:31:16.918787 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:16.918757 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs podName:99da651c-7b27-49ea-b49a-7f72caf21100 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:20.918740031 +0000 UTC m=+10.062043259 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs") pod "network-metrics-daemon-6dxjk" (UID: "99da651c-7b27-49ea-b49a-7f72caf21100") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 18:31:16.919295 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:16.919177 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 18:31:16.919295 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:16.919232 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret podName:25dec05c-c81f-4c62-a6e8-1d3506812ff5 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:20.919216535 +0000 UTC m=+10.062519759 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret") pod "global-pull-secret-syncer-mm2nc" (UID: "25dec05c-c81f-4c62-a6e8-1d3506812ff5") : object "kube-system"/"original-pull-secret" not registered Sep 10 18:31:17.019946 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:17.019275 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gr9s\" (UniqueName: \"kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s\") pod \"network-check-target-dnfs8\" (UID: \"4f84d0cb-448c-4000-85cf-68e4e014c8ec\") " pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:17.019946 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:17.019502 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 18:31:17.019946 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:17.019527 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 18:31:17.019946 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:17.019539 2564 projected.go:196] Error preparing data for projected volume kube-api-access-8gr9s for pod openshift-network-diagnostics/network-check-target-dnfs8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 18:31:17.019946 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:17.019624 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s podName:4f84d0cb-448c-4000-85cf-68e4e014c8ec nodeName:}" failed. No retries permitted until 2026-09-10 18:31:21.019604139 +0000 UTC m=+10.162907385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8gr9s" (UniqueName: "kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s") pod "network-check-target-dnfs8" (UID: "4f84d0cb-448c-4000-85cf-68e4e014c8ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 18:31:17.442688 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:17.442656 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:17.442869 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:17.442803 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:17.443606 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:17.443184 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:17.443606 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:17.443286 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:17.443606 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:17.443446 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:17.443606 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:17.443550 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:17.524136 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:17.524094 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal" event={"ID":"40134ba3305c34e71775d79f7ccc18ed","Type":"ContainerStarted","Data":"6860c1a7cf6063bf8c50f37a3ca9a8f258e04415bcae75b654990b0de7464a6a"} Sep 10 18:31:17.539347 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:17.539293 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-87.ec2.internal" podStartSLOduration=5.539273913 podStartE2EDuration="5.539273913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:31:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:31:17.538219864 +0000 UTC m=+6.681523111" watchObservedRunningTime="2026-09-10 18:31:17.539273913 +0000 UTC m=+6.682577174" Sep 10 18:31:19.441413 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:19.441377 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:19.441824 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:19.441397 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:19.441824 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:19.441550 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:19.441824 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:19.441588 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:19.441824 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:19.441704 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:19.441824 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:19.441789 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:20.951443 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:20.951204 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret\") pod \"global-pull-secret-syncer-mm2nc\" (UID: \"25dec05c-c81f-4c62-a6e8-1d3506812ff5\") " pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:20.951893 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:20.951470 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs\") pod \"network-metrics-daemon-6dxjk\" (UID: \"99da651c-7b27-49ea-b49a-7f72caf21100\") " pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:20.951893 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:20.951351 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 18:31:20.951893 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:20.951574 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret podName:25dec05c-c81f-4c62-a6e8-1d3506812ff5 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:28.951551744 +0000 UTC m=+18.094854985 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret") pod "global-pull-secret-syncer-mm2nc" (UID: "25dec05c-c81f-4c62-a6e8-1d3506812ff5") : object "kube-system"/"original-pull-secret" not registered Sep 10 18:31:20.951893 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:20.951609 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 18:31:20.951893 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:20.951646 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs podName:99da651c-7b27-49ea-b49a-7f72caf21100 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:28.951634292 +0000 UTC m=+18.094937514 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs") pod "network-metrics-daemon-6dxjk" (UID: "99da651c-7b27-49ea-b49a-7f72caf21100") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 18:31:21.053175 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:21.052544 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gr9s\" (UniqueName: \"kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s\") pod \"network-check-target-dnfs8\" (UID: \"4f84d0cb-448c-4000-85cf-68e4e014c8ec\") " pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:21.053175 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:21.052711 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 18:31:21.053175 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:21.052730 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 18:31:21.053175 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:21.052743 2564 projected.go:196] Error preparing data for projected volume kube-api-access-8gr9s for pod openshift-network-diagnostics/network-check-target-dnfs8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 18:31:21.053175 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:21.052800 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s podName:4f84d0cb-448c-4000-85cf-68e4e014c8ec nodeName:}" failed. No retries permitted until 2026-09-10 18:31:29.052780924 +0000 UTC m=+18.196084149 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8gr9s" (UniqueName: "kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s") pod "network-check-target-dnfs8" (UID: "4f84d0cb-448c-4000-85cf-68e4e014c8ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 18:31:21.444416 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:21.443597 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:21.444416 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:21.443750 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:21.444416 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:21.444086 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:21.444416 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:21.444183 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:21.444416 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:21.444230 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:21.444416 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:21.444360 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:23.443155 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:23.443095 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:23.443611 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:23.443095 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:23.443611 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:23.443242 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:23.443611 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:23.443299 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:23.443611 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:23.443419 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:23.443611 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:23.443526 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:25.441770 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:25.441739 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:25.441770 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:25.441770 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:25.442235 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:25.441739 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:25.442235 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:25.441853 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:25.442235 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:25.441965 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:25.442235 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:25.442063 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:27.442025 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:27.441991 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:27.442499 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:27.441996 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:27.442499 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:27.442128 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:27.442499 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:27.442215 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:27.442499 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:27.441999 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:27.442499 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:27.442305 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:29.011915 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:29.011874 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret\") pod \"global-pull-secret-syncer-mm2nc\" (UID: \"25dec05c-c81f-4c62-a6e8-1d3506812ff5\") " pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:29.011915 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:29.011920 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs\") pod \"network-metrics-daemon-6dxjk\" (UID: \"99da651c-7b27-49ea-b49a-7f72caf21100\") " pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:29.012439 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:29.012054 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 18:31:29.012439 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:29.012054 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 18:31:29.012439 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:29.012117 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs podName:99da651c-7b27-49ea-b49a-7f72caf21100 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:45.012097944 +0000 UTC m=+34.155401185 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs") pod "network-metrics-daemon-6dxjk" (UID: "99da651c-7b27-49ea-b49a-7f72caf21100") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 18:31:29.012439 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:29.012141 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret podName:25dec05c-c81f-4c62-a6e8-1d3506812ff5 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:45.012131032 +0000 UTC m=+34.155434260 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret") pod "global-pull-secret-syncer-mm2nc" (UID: "25dec05c-c81f-4c62-a6e8-1d3506812ff5") : object "kube-system"/"original-pull-secret" not registered Sep 10 18:31:29.113340 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:29.113301 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gr9s\" (UniqueName: \"kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s\") pod \"network-check-target-dnfs8\" (UID: \"4f84d0cb-448c-4000-85cf-68e4e014c8ec\") " pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:29.113562 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:29.113432 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 18:31:29.113562 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:29.113452 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 18:31:29.113562 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:29.113461 2564 projected.go:196] Error preparing data for projected volume kube-api-access-8gr9s for pod openshift-network-diagnostics/network-check-target-dnfs8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 18:31:29.113562 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:29.113532 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s podName:4f84d0cb-448c-4000-85cf-68e4e014c8ec nodeName:}" failed. No retries permitted until 2026-09-10 18:31:45.113516483 +0000 UTC m=+34.256819708 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8gr9s" (UniqueName: "kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s") pod "network-check-target-dnfs8" (UID: "4f84d0cb-448c-4000-85cf-68e4e014c8ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 18:31:29.441816 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:29.441731 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:29.441978 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:29.441731 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:29.441978 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:29.441871 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:29.441978 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:29.441880 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:29.441978 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:29.441965 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:29.442187 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:29.442049 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:31.442942 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:31.442896 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:31.443409 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:31.442996 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:31.443409 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:31.443027 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:31.443409 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:31.443047 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:31.443409 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:31.443106 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:31.443409 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:31.443171 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:32.557128 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.556959 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7h88j" event={"ID":"b0b66f99-0213-4038-9319-be325dc2afad","Type":"ContainerStarted","Data":"5ccff10215ccc4239efeddc3696db9a155a7fcbc38f08f6396f728ece5392373"} Sep 10 18:31:32.559762 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.559743 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/ovn-acl-logging/0.log" Sep 10 18:31:32.560194 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.560175 2564 generic.go:356] "Generic (PLEG): container finished" podID="019680e6-9104-47fb-b272-9ced99c8bfc8" containerID="561bc71b7b5b27bdfccb65c8374f04f6e5a85c0536a440b54b4e9158b39bb22b" exitCode=1 Sep 10 18:31:32.560266 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.560252 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" event={"ID":"019680e6-9104-47fb-b272-9ced99c8bfc8","Type":"ContainerStarted","Data":"f12a2579f9d65c05b52eeebae2e40deb08a6de372a2abb0b31050914fdff1659"} Sep 10 18:31:32.560321 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.560304 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" event={"ID":"019680e6-9104-47fb-b272-9ced99c8bfc8","Type":"ContainerStarted","Data":"ddaca5fcc8c552882a2d3056bb3d5a1e3c08116c583abca482caa358c98e2ab9"} Sep 10 18:31:32.560365 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.560351 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" event={"ID":"019680e6-9104-47fb-b272-9ced99c8bfc8","Type":"ContainerStarted","Data":"753ec553184591e71f8c36926b9aba24b16e832add7074ba78ddbf3479b358f4"} Sep 10 18:31:32.560405 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.560396 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" event={"ID":"019680e6-9104-47fb-b272-9ced99c8bfc8","Type":"ContainerStarted","Data":"a136e99ae047563c686a322513c7aa96e2d0a4b95ff3bd6303b040b5307745bc"} Sep 10 18:31:32.560438 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.560427 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" event={"ID":"019680e6-9104-47fb-b272-9ced99c8bfc8","Type":"ContainerDied","Data":"561bc71b7b5b27bdfccb65c8374f04f6e5a85c0536a440b54b4e9158b39bb22b"} Sep 10 18:31:32.560497 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.560469 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" event={"ID":"019680e6-9104-47fb-b272-9ced99c8bfc8","Type":"ContainerStarted","Data":"ca67d3fa1547761d744daca569dccdb5da6a87e22b737ebb974d2a330c3748ae"} Sep 10 18:31:32.561777 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.561753 2564 generic.go:356] "Generic (PLEG): container finished" podID="92d1a4d6-3b6a-4676-b798-8ca176789fb1" containerID="45bc75d162621b73b36f79810dd69ab4d5b1d25c4f93bf4ec37fc85467ebd422" exitCode=0 Sep 10 18:31:32.561859 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.561821 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nrlp8" event={"ID":"92d1a4d6-3b6a-4676-b798-8ca176789fb1","Type":"ContainerDied","Data":"45bc75d162621b73b36f79810dd69ab4d5b1d25c4f93bf4ec37fc85467ebd422"} Sep 10 18:31:32.563301 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.563276 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4lr7x" event={"ID":"e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0","Type":"ContainerStarted","Data":"d7387ee39d97e3df25a82f30911696cf3a3ca540827f481674fd97afd4a5e72f"} Sep 10 18:31:32.564843 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.564823 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" event={"ID":"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4","Type":"ContainerStarted","Data":"e9388f44d9c285157fa5242f96b2dc611665ead4c2c3bb615f231ffb421dc1b5"} Sep 10 18:31:32.566135 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.566114 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cd2ls" event={"ID":"96332d76-e827-4612-a6b5-bd9d31fe347e","Type":"ContainerStarted","Data":"fb151366eb138e580c624f23cd7e51912e83ecc63c28424606b5d633b726a370"} Sep 10 18:31:32.567581 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.567551 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q4chw" event={"ID":"97385966-6ff5-44b8-9a49-cfb01d67fefe","Type":"ContainerStarted","Data":"db7a705289cee10d813fbe4a888d680813dc2a846875a33bc0738d771abe1a4e"} Sep 10 18:31:32.571947 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.571921 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6jw7x" event={"ID":"3e20510c-c392-4fc1-a566-e2b9179d3e08","Type":"ContainerStarted","Data":"22ba0221054004c0138323f6181697422375542fa140e5a5a861835ebfedfca8"} Sep 10 18:31:32.584992 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.584951 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7h88j" podStartSLOduration=3.439665463 podStartE2EDuration="21.584938473s" totalImagesPullingTime="18.14527301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:31:32.572243518 +0000 UTC m=+21.715546781" watchObservedRunningTime="2026-09-10 18:31:32.584938473 +0000 UTC m=+21.728241717" Sep 10 18:31:32.585078 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.585053 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cd2ls" podStartSLOduration=3.443291783 podStartE2EDuration="21.585030571s" totalImagesPullingTime="18.141738788s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:31:32.58451897 +0000 UTC m=+21.727822233" watchObservedRunningTime="2026-09-10 18:31:32.585030571 +0000 UTC m=+21.728333816" Sep 10 18:31:32.632137 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.632094 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6jw7x" podStartSLOduration=3.129062968 podStartE2EDuration="21.632078855s" totalImagesPullingTime="18.503015887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:31:32.631598757 +0000 UTC m=+21.774902025" watchObservedRunningTime="2026-09-10 18:31:32.632078855 +0000 UTC m=+21.775382100" Sep 10 18:31:32.645682 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.645645 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4lr7x" podStartSLOduration=3.520484206 podStartE2EDuration="21.645631578s" totalImagesPullingTime="18.125147372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:31:32.645442696 +0000 UTC m=+21.788745969" watchObservedRunningTime="2026-09-10 18:31:32.645631578 +0000 UTC m=+21.788934821" Sep 10 18:31:32.658185 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:32.658133 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-q4chw" podStartSLOduration=7.612876395 podStartE2EDuration="21.658114122s" totalImagesPullingTime="14.045237727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:31:32.657764513 +0000 UTC m=+21.801067757" watchObservedRunningTime="2026-09-10 18:31:32.658114122 +0000 UTC m=+21.801417367" Sep 10 18:31:33.441387 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:33.441354 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:33.441624 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:33.441354 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:33.441624 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:33.441503 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:33.441624 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:33.441583 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:33.441624 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:33.441364 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:33.441828 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:33.441687 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:33.576476 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:33.576398 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6thwj" event={"ID":"d0a0a9fe-c357-48a7-ac8d-086772b736d8","Type":"ContainerStarted","Data":"634da6c6253033d4265ad35861b6dd5e8628a6c2f753ddc1f8162dfd23e76fc3"} Sep 10 18:31:33.590282 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:33.590235 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6thwj" podStartSLOduration=4.408995583 podStartE2EDuration="22.590221166s" totalImagesPullingTime="18.181225583s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:31:33.589985696 +0000 UTC m=+22.733288936" watchObservedRunningTime="2026-09-10 18:31:33.590221166 +0000 UTC m=+22.733524411" Sep 10 18:31:33.999144 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:33.998938 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 18:31:34.485955 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:34.485867 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T18:31:33.999133025Z","UUID":"23771860-fcdc-4054-b13b-599c6de30998","Handler":null,"Name":"","Endpoint":""} Sep 10 18:31:34.487194 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:34.487175 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 18:31:34.487327 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:34.487205 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 18:31:34.582951 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:34.582921 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" event={"ID":"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4","Type":"ContainerStarted","Data":"18093a5945b50dcfcf6884bd56cf22312f8c9365da7ac74371ef9549607f5c0b"} Sep 10 18:31:35.442323 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:35.442241 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:35.442480 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:35.442241 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:35.442480 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:35.442373 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:35.442480 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:35.442394 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:35.442643 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:35.442539 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:35.442643 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:35.442626 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:35.587414 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:35.587381 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/ovn-acl-logging/0.log" Sep 10 18:31:35.587832 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:35.587808 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" event={"ID":"019680e6-9104-47fb-b272-9ced99c8bfc8","Type":"ContainerStarted","Data":"eb10e9c2106bd8cd4a3415d74da2f8562fa50b8677efe501bec6eb2b0f7f6196"} Sep 10 18:31:35.589776 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:35.589759 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" event={"ID":"f1a1c859-d4ec-4480-80c6-6d5cc5f3d1d4","Type":"ContainerStarted","Data":"0dce7b8e3ebcc6cc975a8d86d49b061db18cf8ebcb6ada2ada292ea232836393"} Sep 10 18:31:35.619509 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:35.619456 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mvn4p" podStartSLOduration=3.451574914 podStartE2EDuration="24.619444999s" totalImagesPullingTime="21.167870085s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:31:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:31:35.618997203 +0000 UTC m=+24.762300447" watchObservedRunningTime="2026-09-10 18:31:35.619444999 +0000 UTC m=+24.762748240" Sep 10 18:31:37.396631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:37.396402 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-q4chw" Sep 10 18:31:37.397076 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:37.396907 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-q4chw" Sep 10 18:31:37.441826 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:37.441792 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:37.441826 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:37.441826 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:37.442043 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:37.441797 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:37.442043 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:37.441885 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:37.442043 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:37.441975 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:37.442197 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:37.442046 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:37.598609 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:37.598580 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/ovn-acl-logging/0.log" Sep 10 18:31:37.599072 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:37.599041 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" event={"ID":"019680e6-9104-47fb-b272-9ced99c8bfc8","Type":"ContainerStarted","Data":"d33e8568d05484deadb34dfc80d98051555a25596a27b4cd8798257567af349c"} Sep 10 18:31:37.599292 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:37.599270 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-q4chw" Sep 10 18:31:37.599607 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:37.599574 2564 scope.go:117] "RemoveContainer" containerID="561bc71b7b5b27bdfccb65c8374f04f6e5a85c0536a440b54b4e9158b39bb22b" Sep 10 18:31:37.599817 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:37.599798 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-q4chw" Sep 10 18:31:38.721053 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:38.721020 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dnfs8"] Sep 10 18:31:38.721503 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:38.721190 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:38.721503 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:38.721297 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:38.723768 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:38.723744 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6dxjk"] Sep 10 18:31:38.723903 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:38.723853 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:38.723977 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:38.723955 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:38.731422 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:38.731397 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mm2nc"] Sep 10 18:31:38.731530 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:38.731501 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:38.731597 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:38.731575 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:39.608040 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:39.608006 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/ovn-acl-logging/0.log" Sep 10 18:31:39.608503 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:39.608451 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" event={"ID":"019680e6-9104-47fb-b272-9ced99c8bfc8","Type":"ContainerStarted","Data":"73900a7a2970043d51410091b5a7375b48cabd61449d44fb31fb9f9ad672b609"} Sep 10 18:31:39.608872 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:39.608848 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:39.608996 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:39.608925 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:39.608996 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:39.608979 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:39.610555 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:39.610528 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nrlp8" event={"ID":"92d1a4d6-3b6a-4676-b798-8ca176789fb1","Type":"ContainerStarted","Data":"49e1348ae6b51f919dd98ba3c850fe9e2e1372858bb4425e84fcc44acc861d10"} Sep 10 18:31:39.624123 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:39.624094 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:39.624241 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:39.624224 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:31:39.635068 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:39.635001 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" podStartSLOduration=10.427228741 podStartE2EDuration="28.634979857s" totalImagesPullingTime="18.207751116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:31:39.633626933 +0000 UTC m=+28.776930176" watchObservedRunningTime="2026-09-10 18:31:39.634979857 +0000 UTC m=+28.778283102" Sep 10 18:31:40.441967 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:40.441942 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:40.442476 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:40.441945 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:40.442476 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:40.442049 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:40.442476 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:40.442081 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:40.442476 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:40.442176 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:40.442476 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:40.442250 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:40.614457 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:40.614417 2564 generic.go:356] "Generic (PLEG): container finished" podID="92d1a4d6-3b6a-4676-b798-8ca176789fb1" containerID="49e1348ae6b51f919dd98ba3c850fe9e2e1372858bb4425e84fcc44acc861d10" exitCode=0 Sep 10 18:31:40.614651 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:40.614512 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nrlp8" event={"ID":"92d1a4d6-3b6a-4676-b798-8ca176789fb1","Type":"ContainerDied","Data":"49e1348ae6b51f919dd98ba3c850fe9e2e1372858bb4425e84fcc44acc861d10"} Sep 10 18:31:41.629601 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:41.629442 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" podUID="019680e6-9104-47fb-b272-9ced99c8bfc8" containerName="ovnkube-controller" probeResult="failure" output="" Sep 10 18:31:42.441605 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:42.441567 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:42.441773 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:42.441668 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:42.441856 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:42.441674 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:42.441899 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:42.441670 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:42.441899 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:42.441789 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:42.441899 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:42.441871 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:42.620700 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:42.620666 2564 generic.go:356] "Generic (PLEG): container finished" podID="92d1a4d6-3b6a-4676-b798-8ca176789fb1" containerID="0831e419a2a769af4cbe25b27ac8b0cc3b094868649e68bbbadc255f1135c0c1" exitCode=0 Sep 10 18:31:42.620831 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:42.620714 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nrlp8" event={"ID":"92d1a4d6-3b6a-4676-b798-8ca176789fb1","Type":"ContainerDied","Data":"0831e419a2a769af4cbe25b27ac8b0cc3b094868649e68bbbadc255f1135c0c1"} Sep 10 18:31:43.625755 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:43.625676 2564 generic.go:356] "Generic (PLEG): container finished" podID="92d1a4d6-3b6a-4676-b798-8ca176789fb1" containerID="c133e7d8f42e44df2cb3dea4ad6aa52138c601d5d96215cfb7a013b2c3b21a2c" exitCode=0 Sep 10 18:31:43.625755 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:43.625726 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nrlp8" event={"ID":"92d1a4d6-3b6a-4676-b798-8ca176789fb1","Type":"ContainerDied","Data":"c133e7d8f42e44df2cb3dea4ad6aa52138c601d5d96215cfb7a013b2c3b21a2c"} Sep 10 18:31:44.441292 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.441260 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:44.441445 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.441392 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:44.441445 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:44.441399 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mm2nc" podUID="25dec05c-c81f-4c62-a6e8-1d3506812ff5" Sep 10 18:31:44.441445 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.441421 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:44.441613 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:44.441532 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:31:44.441613 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:44.441596 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dnfs8" podUID="4f84d0cb-448c-4000-85cf-68e4e014c8ec" Sep 10 18:31:44.725736 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.725660 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-87.ec2.internal" event="NodeReady" Sep 10 18:31:44.726162 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.725812 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 18:31:44.760390 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.760358 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-5tq6l"] Sep 10 18:31:44.764079 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.764055 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84ccb96897-cld7r"] Sep 10 18:31:44.764079 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.764078 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:31:44.768156 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.767957 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-rmh66\"" Sep 10 18:31:44.768156 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.768033 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 10 18:31:44.768758 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.768735 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 10 18:31:44.770106 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.770084 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:44.771674 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.771656 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-5tq6l"] Sep 10 18:31:44.773091 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.773076 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 10 18:31:44.773310 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.773294 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xgqmb\"" Sep 10 18:31:44.773449 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.773427 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 10 18:31:44.773568 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.773510 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 10 18:31:44.776273 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.776250 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zkgwb"] Sep 10 18:31:44.780205 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.780183 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 10 18:31:44.780304 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.780292 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zkgwb" Sep 10 18:31:44.780455 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.780436 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nggdz"] Sep 10 18:31:44.782312 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.782282 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 18:31:44.782451 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.782285 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 18:31:44.782690 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.782671 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 18:31:44.783283 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.782979 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-h28kp\"" Sep 10 18:31:44.784365 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.784337 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nggdz" Sep 10 18:31:44.784462 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.784446 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84ccb96897-cld7r"] Sep 10 18:31:44.786235 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.786211 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 18:31:44.786457 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.786441 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4lj58\"" Sep 10 18:31:44.786631 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.786615 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 18:31:44.794893 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.794866 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nggdz"] Sep 10 18:31:44.797407 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.797369 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zkgwb"] Sep 10 18:31:44.825380 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.825348 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5tq6l\" (UID: \"c6c62a12-04eb-40ce-b5cb-3a43fc562884\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:31:44.825558 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.825393 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c6c62a12-04eb-40ce-b5cb-3a43fc562884-nginx-conf\") pod \"networking-console-plugin-79977f9844-5tq6l\" (UID: \"c6c62a12-04eb-40ce-b5cb-3a43fc562884\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:31:44.925834 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.925800 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/857e07ef-d71a-41f6-b863-b90126b0febb-config-volume\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:31:44.926036 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.925842 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/857e07ef-d71a-41f6-b863-b90126b0febb-tmp-dir\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:31:44.926036 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.925870 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-ca-trust-extracted\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:44.926036 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.925899 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-installation-pull-secrets\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:44.926036 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.925940 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c6c62a12-04eb-40ce-b5cb-3a43fc562884-nginx-conf\") pod \"networking-console-plugin-79977f9844-5tq6l\" (UID: \"c6c62a12-04eb-40ce-b5cb-3a43fc562884\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:31:44.926036 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.925994 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:31:44.926036 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.926029 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qll8\" (UniqueName: \"kubernetes.io/projected/cc9720ca-244c-4b07-88e5-e95fbeae172d-kube-api-access-5qll8\") pod \"ingress-canary-zkgwb\" (UID: \"cc9720ca-244c-4b07-88e5-e95fbeae172d\") " pod="openshift-ingress-canary/ingress-canary-zkgwb" Sep 10 18:31:44.926285 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.926057 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc9720ca-244c-4b07-88e5-e95fbeae172d-cert\") pod \"ingress-canary-zkgwb\" (UID: \"cc9720ca-244c-4b07-88e5-e95fbeae172d\") " pod="openshift-ingress-canary/ingress-canary-zkgwb" Sep 10 18:31:44.926285 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.926153 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-image-registry-private-configuration\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:44.926285 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.926185 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:44.926285 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.926201 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-certificates\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:44.926285 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.926229 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5tq6l\" (UID: \"c6c62a12-04eb-40ce-b5cb-3a43fc562884\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:31:44.926285 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.926267 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-trusted-ca\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:44.926590 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:44.926307 2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 18:31:44.926590 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.926306 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb8kp\" (UniqueName: \"kubernetes.io/projected/857e07ef-d71a-41f6-b863-b90126b0febb-kube-api-access-bb8kp\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:31:44.926590 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:44.926376 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert podName:c6c62a12-04eb-40ce-b5cb-3a43fc562884 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:45.426354021 +0000 UTC m=+34.569657264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5tq6l" (UID: "c6c62a12-04eb-40ce-b5cb-3a43fc562884") : secret "networking-console-plugin-cert" not found Sep 10 18:31:44.926590 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.926415 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-bound-sa-token\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:44.926590 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.926439 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l4s9\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-kube-api-access-8l4s9\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:44.926771 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:44.926664 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c6c62a12-04eb-40ce-b5cb-3a43fc562884-nginx-conf\") pod \"networking-console-plugin-79977f9844-5tq6l\" (UID: \"c6c62a12-04eb-40ce-b5cb-3a43fc562884\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:31:45.027522 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.027424 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:31:45.027522 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.027468 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qll8\" (UniqueName: \"kubernetes.io/projected/cc9720ca-244c-4b07-88e5-e95fbeae172d-kube-api-access-5qll8\") pod \"ingress-canary-zkgwb\" (UID: \"cc9720ca-244c-4b07-88e5-e95fbeae172d\") " pod="openshift-ingress-canary/ingress-canary-zkgwb" Sep 10 18:31:45.027522 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.027505 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc9720ca-244c-4b07-88e5-e95fbeae172d-cert\") pod \"ingress-canary-zkgwb\" (UID: \"cc9720ca-244c-4b07-88e5-e95fbeae172d\") " pod="openshift-ingress-canary/ingress-canary-zkgwb" Sep 10 18:31:45.027781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.027565 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret\") pod \"global-pull-secret-syncer-mm2nc\" (UID: \"25dec05c-c81f-4c62-a6e8-1d3506812ff5\") " pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:45.027781 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.027588 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 18:31:45.027781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.027605 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs\") pod \"network-metrics-daemon-6dxjk\" (UID: \"99da651c-7b27-49ea-b49a-7f72caf21100\") " pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:45.027781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.027626 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-image-registry-private-configuration\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:45.027781 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.027660 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls podName:857e07ef-d71a-41f6-b863-b90126b0febb nodeName:}" failed. No retries permitted until 2026-09-10 18:31:45.527637646 +0000 UTC m=+34.670940885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls") pod "dns-default-nggdz" (UID: "857e07ef-d71a-41f6-b863-b90126b0febb") : secret "dns-default-metrics-tls" not found Sep 10 18:31:45.027781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.027697 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:45.027781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.027724 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-certificates\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:45.027781 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.027729 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 18:31:45.027781 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.027761 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 18:31:45.027781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.027773 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-trusted-ca\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:45.028256 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.027795 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret podName:25dec05c-c81f-4c62-a6e8-1d3506812ff5 nodeName:}" failed. No retries permitted until 2026-09-10 18:32:17.027779992 +0000 UTC m=+66.171083231 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret") pod "global-pull-secret-syncer-mm2nc" (UID: "25dec05c-c81f-4c62-a6e8-1d3506812ff5") : object "kube-system"/"original-pull-secret" not registered Sep 10 18:31:45.028256 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.027801 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 18:31:45.028256 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.027816 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84ccb96897-cld7r: secret "image-registry-tls" not found Sep 10 18:31:45.028256 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.027825 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs podName:99da651c-7b27-49ea-b49a-7f72caf21100 nodeName:}" failed. No retries permitted until 2026-09-10 18:32:17.02780726 +0000 UTC m=+66.171110504 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs") pod "network-metrics-daemon-6dxjk" (UID: "99da651c-7b27-49ea-b49a-7f72caf21100") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 18:31:45.028256 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.027860 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb8kp\" (UniqueName: \"kubernetes.io/projected/857e07ef-d71a-41f6-b863-b90126b0febb-kube-api-access-bb8kp\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:31:45.028256 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.027873 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls podName:05dbe2c5-05cf-4b20-99e2-a50b46fe0252 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:45.527860317 +0000 UTC m=+34.671163550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls") pod "image-registry-84ccb96897-cld7r" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252") : secret "image-registry-tls" not found Sep 10 18:31:45.028256 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.027904 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-bound-sa-token\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:45.028256 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.027940 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l4s9\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-kube-api-access-8l4s9\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:45.028256 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.027970 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/857e07ef-d71a-41f6-b863-b90126b0febb-config-volume\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:31:45.028256 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.028001 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/857e07ef-d71a-41f6-b863-b90126b0febb-tmp-dir\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:31:45.028256 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.028032 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-ca-trust-extracted\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:45.028256 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.028065 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-installation-pull-secrets\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:45.028817 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.028319 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-certificates\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:45.028817 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.028462 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/857e07ef-d71a-41f6-b863-b90126b0febb-tmp-dir\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:31:45.028817 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.028608 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/857e07ef-d71a-41f6-b863-b90126b0febb-config-volume\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:31:45.028817 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.028742 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-ca-trust-extracted\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:45.029249 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.029229 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-trusted-ca\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:45.031657 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.031635 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-image-registry-private-configuration\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:45.031657 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.031644 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-installation-pull-secrets\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:45.031795 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.031646 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc9720ca-244c-4b07-88e5-e95fbeae172d-cert\") pod \"ingress-canary-zkgwb\" (UID: \"cc9720ca-244c-4b07-88e5-e95fbeae172d\") " pod="openshift-ingress-canary/ingress-canary-zkgwb" Sep 10 18:31:45.035526 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.035497 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb8kp\" (UniqueName: \"kubernetes.io/projected/857e07ef-d71a-41f6-b863-b90126b0febb-kube-api-access-bb8kp\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:31:45.035676 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.035654 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-bound-sa-token\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:45.035724 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.035689 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l4s9\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-kube-api-access-8l4s9\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:45.036109 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.036079 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qll8\" (UniqueName: \"kubernetes.io/projected/cc9720ca-244c-4b07-88e5-e95fbeae172d-kube-api-access-5qll8\") pod \"ingress-canary-zkgwb\" (UID: \"cc9720ca-244c-4b07-88e5-e95fbeae172d\") " pod="openshift-ingress-canary/ingress-canary-zkgwb" Sep 10 18:31:45.096374 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.096352 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zkgwb" Sep 10 18:31:45.129554 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.129308 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gr9s\" (UniqueName: \"kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s\") pod \"network-check-target-dnfs8\" (UID: \"4f84d0cb-448c-4000-85cf-68e4e014c8ec\") " pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:45.129681 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.129503 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 18:31:45.129681 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.129617 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 18:31:45.129681 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.129635 2564 projected.go:196] Error preparing data for projected volume kube-api-access-8gr9s for pod openshift-network-diagnostics/network-check-target-dnfs8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 18:31:45.129802 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.129702 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s podName:4f84d0cb-448c-4000-85cf-68e4e014c8ec nodeName:}" failed. No retries permitted until 2026-09-10 18:32:17.129682986 +0000 UTC m=+66.272986222 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8gr9s" (UniqueName: "kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s") pod "network-check-target-dnfs8" (UID: "4f84d0cb-448c-4000-85cf-68e4e014c8ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 18:31:45.381625 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.381527 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zkgwb"] Sep 10 18:31:45.386177 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:31:45.386108 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc9720ca_244c_4b07_88e5_e95fbeae172d.slice/crio-7c8d3c2d05a2a40a818180ce02637156fca23cd91c7cdde2467961fa3bd78030 WatchSource:0}: Error finding container 7c8d3c2d05a2a40a818180ce02637156fca23cd91c7cdde2467961fa3bd78030: Status 404 returned error can't find the container with id 7c8d3c2d05a2a40a818180ce02637156fca23cd91c7cdde2467961fa3bd78030 Sep 10 18:31:45.431816 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.431786 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5tq6l\" (UID: \"c6c62a12-04eb-40ce-b5cb-3a43fc562884\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:31:45.431976 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.431945 2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 18:31:45.432092 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.432025 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert podName:c6c62a12-04eb-40ce-b5cb-3a43fc562884 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:46.432002845 +0000 UTC m=+35.575306081 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5tq6l" (UID: "c6c62a12-04eb-40ce-b5cb-3a43fc562884") : secret "networking-console-plugin-cert" not found Sep 10 18:31:45.532846 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.532813 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:45.533008 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.532879 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:31:45.533008 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.532964 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 18:31:45.533008 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.532971 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 18:31:45.533008 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.532990 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84ccb96897-cld7r: secret "image-registry-tls" not found Sep 10 18:31:45.533152 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.533013 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls podName:857e07ef-d71a-41f6-b863-b90126b0febb nodeName:}" failed. No retries permitted until 2026-09-10 18:31:46.533001401 +0000 UTC m=+35.676304624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls") pod "dns-default-nggdz" (UID: "857e07ef-d71a-41f6-b863-b90126b0febb") : secret "dns-default-metrics-tls" not found Sep 10 18:31:45.533152 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:45.533038 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls podName:05dbe2c5-05cf-4b20-99e2-a50b46fe0252 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:46.533024061 +0000 UTC m=+35.676327299 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls") pod "image-registry-84ccb96897-cld7r" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252") : secret "image-registry-tls" not found Sep 10 18:31:45.632564 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:45.632497 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zkgwb" event={"ID":"cc9720ca-244c-4b07-88e5-e95fbeae172d","Type":"ContainerStarted","Data":"7c8d3c2d05a2a40a818180ce02637156fca23cd91c7cdde2467961fa3bd78030"} Sep 10 18:31:46.440168 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:46.440128 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5tq6l\" (UID: \"c6c62a12-04eb-40ce-b5cb-3a43fc562884\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:31:46.440757 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:46.440366 2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 18:31:46.440757 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:46.440445 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert podName:c6c62a12-04eb-40ce-b5cb-3a43fc562884 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:48.440426388 +0000 UTC m=+37.583729616 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5tq6l" (UID: "c6c62a12-04eb-40ce-b5cb-3a43fc562884") : secret "networking-console-plugin-cert" not found Sep 10 18:31:46.442286 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:46.442206 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:31:46.442423 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:46.442318 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:31:46.442511 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:46.442467 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:31:46.444574 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:46.444552 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 18:31:46.444689 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:46.444560 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 18:31:46.445133 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:46.445115 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 18:31:46.445227 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:46.445172 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dxnbp\"" Sep 10 18:31:46.445273 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:46.445115 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 18:31:46.445428 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:46.445361 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-thvkz\"" Sep 10 18:31:46.541416 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:46.540908 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:46.541416 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:46.541016 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:31:46.541416 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:46.541041 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 18:31:46.541416 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:46.541059 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84ccb96897-cld7r: secret "image-registry-tls" not found Sep 10 18:31:46.541416 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:46.541130 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls podName:05dbe2c5-05cf-4b20-99e2-a50b46fe0252 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:48.541111434 +0000 UTC m=+37.684414668 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls") pod "image-registry-84ccb96897-cld7r" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252") : secret "image-registry-tls" not found Sep 10 18:31:46.541820 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:46.541562 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 18:31:46.541820 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:46.541648 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls podName:857e07ef-d71a-41f6-b863-b90126b0febb nodeName:}" failed. No retries permitted until 2026-09-10 18:31:48.54163223 +0000 UTC m=+37.684935456 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls") pod "dns-default-nggdz" (UID: "857e07ef-d71a-41f6-b863-b90126b0febb") : secret "dns-default-metrics-tls" not found Sep 10 18:31:48.455786 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:48.455749 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5tq6l\" (UID: \"c6c62a12-04eb-40ce-b5cb-3a43fc562884\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:31:48.456337 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:48.455900 2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 18:31:48.456337 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:48.455958 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert podName:c6c62a12-04eb-40ce-b5cb-3a43fc562884 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:52.455943355 +0000 UTC m=+41.599246599 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5tq6l" (UID: "c6c62a12-04eb-40ce-b5cb-3a43fc562884") : secret "networking-console-plugin-cert" not found Sep 10 18:31:48.556674 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:48.556593 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:31:48.556850 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:48.556690 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:48.556850 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:48.556760 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 18:31:48.556850 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:48.556827 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 18:31:48.556850 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:48.556844 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84ccb96897-cld7r: secret "image-registry-tls" not found Sep 10 18:31:48.557054 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:48.556830 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls podName:857e07ef-d71a-41f6-b863-b90126b0febb nodeName:}" failed. No retries permitted until 2026-09-10 18:31:52.556809436 +0000 UTC m=+41.700112672 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls") pod "dns-default-nggdz" (UID: "857e07ef-d71a-41f6-b863-b90126b0febb") : secret "dns-default-metrics-tls" not found Sep 10 18:31:48.557054 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:48.556901 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls podName:05dbe2c5-05cf-4b20-99e2-a50b46fe0252 nodeName:}" failed. No retries permitted until 2026-09-10 18:31:52.556885721 +0000 UTC m=+41.700188956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls") pod "image-registry-84ccb96897-cld7r" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252") : secret "image-registry-tls" not found Sep 10 18:31:50.648989 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:50.648946 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zkgwb" event={"ID":"cc9720ca-244c-4b07-88e5-e95fbeae172d","Type":"ContainerStarted","Data":"6f6869b588859dc0ff8388b3139f2f9b014ba24ea1c61c224718e352e4bb3aef"} Sep 10 18:31:50.651710 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:50.651681 2564 generic.go:356] "Generic (PLEG): container finished" podID="92d1a4d6-3b6a-4676-b798-8ca176789fb1" containerID="33a83cac29ee38e3c34c7302efeecdcc078b171fbad10d6b12548635b61a94c8" exitCode=0 Sep 10 18:31:50.651826 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:50.651740 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nrlp8" event={"ID":"92d1a4d6-3b6a-4676-b798-8ca176789fb1","Type":"ContainerDied","Data":"33a83cac29ee38e3c34c7302efeecdcc078b171fbad10d6b12548635b61a94c8"} Sep 10 18:31:50.665221 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:50.665181 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zkgwb" podStartSLOduration=1.825133407 podStartE2EDuration="6.665165883s" totalImagesPullingTime="4.840032476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:31:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:31:50.664233357 +0000 UTC m=+39.807536601" watchObservedRunningTime="2026-09-10 18:31:50.665165883 +0000 UTC m=+39.808469141" Sep 10 18:31:51.656839 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:51.656805 2564 generic.go:356] "Generic (PLEG): container finished" podID="92d1a4d6-3b6a-4676-b798-8ca176789fb1" containerID="11a694eb2cdb861964f3c6d15e54f90c83e1d4ddc84b4cede7f003321d9cf099" exitCode=0 Sep 10 18:31:51.657213 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:51.656884 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nrlp8" event={"ID":"92d1a4d6-3b6a-4676-b798-8ca176789fb1","Type":"ContainerDied","Data":"11a694eb2cdb861964f3c6d15e54f90c83e1d4ddc84b4cede7f003321d9cf099"} Sep 10 18:31:52.488164 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:52.488120 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5tq6l\" (UID: \"c6c62a12-04eb-40ce-b5cb-3a43fc562884\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:31:52.488342 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:52.488290 2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 18:31:52.488380 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:52.488359 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert podName:c6c62a12-04eb-40ce-b5cb-3a43fc562884 nodeName:}" failed. No retries permitted until 2026-09-10 18:32:00.488343128 +0000 UTC m=+49.631646355 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5tq6l" (UID: "c6c62a12-04eb-40ce-b5cb-3a43fc562884") : secret "networking-console-plugin-cert" not found Sep 10 18:31:52.589498 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:52.589436 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:31:52.589687 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:52.589567 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:31:52.589687 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:52.589576 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 18:31:52.589687 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:52.589644 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls podName:857e07ef-d71a-41f6-b863-b90126b0febb nodeName:}" failed. No retries permitted until 2026-09-10 18:32:00.589622018 +0000 UTC m=+49.732925252 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls") pod "dns-default-nggdz" (UID: "857e07ef-d71a-41f6-b863-b90126b0febb") : secret "dns-default-metrics-tls" not found Sep 10 18:31:52.589791 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:52.589712 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 18:31:52.589791 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:52.589735 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84ccb96897-cld7r: secret "image-registry-tls" not found Sep 10 18:31:52.589852 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:31:52.589792 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls podName:05dbe2c5-05cf-4b20-99e2-a50b46fe0252 nodeName:}" failed. No retries permitted until 2026-09-10 18:32:00.589778858 +0000 UTC m=+49.733082081 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls") pod "image-registry-84ccb96897-cld7r" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252") : secret "image-registry-tls" not found Sep 10 18:31:52.662800 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:52.662764 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nrlp8" event={"ID":"92d1a4d6-3b6a-4676-b798-8ca176789fb1","Type":"ContainerStarted","Data":"71974eee95cc1900442586a15bbfe89c3d6691acd49eb8725170e066cb007180"} Sep 10 18:31:52.683236 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:31:52.683194 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nrlp8" podStartSLOduration=8.330279212 podStartE2EDuration="41.683180585s" totalImagesPullingTime="33.352901373s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:31:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:31:52.682060592 +0000 UTC m=+41.825363846" watchObservedRunningTime="2026-09-10 18:31:52.683180585 +0000 UTC m=+41.826483826" Sep 10 18:32:00.548228 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:00.548189 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5tq6l\" (UID: \"c6c62a12-04eb-40ce-b5cb-3a43fc562884\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:32:00.548738 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:00.548326 2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 18:32:00.548738 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:00.548389 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert podName:c6c62a12-04eb-40ce-b5cb-3a43fc562884 nodeName:}" failed. No retries permitted until 2026-09-10 18:32:16.54837267 +0000 UTC m=+65.691675908 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5tq6l" (UID: "c6c62a12-04eb-40ce-b5cb-3a43fc562884") : secret "networking-console-plugin-cert" not found Sep 10 18:32:00.649582 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:00.649547 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:32:00.649743 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:00.649637 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:32:00.649743 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:00.649687 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 18:32:00.649743 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:00.649744 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls podName:857e07ef-d71a-41f6-b863-b90126b0febb nodeName:}" failed. No retries permitted until 2026-09-10 18:32:16.649729822 +0000 UTC m=+65.793033070 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls") pod "dns-default-nggdz" (UID: "857e07ef-d71a-41f6-b863-b90126b0febb") : secret "dns-default-metrics-tls" not found Sep 10 18:32:00.649882 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:00.649796 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 18:32:00.649882 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:00.649813 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84ccb96897-cld7r: secret "image-registry-tls" not found Sep 10 18:32:00.649882 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:00.649863 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls podName:05dbe2c5-05cf-4b20-99e2-a50b46fe0252 nodeName:}" failed. No retries permitted until 2026-09-10 18:32:16.649849803 +0000 UTC m=+65.793153025 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls") pod "image-registry-84ccb96897-cld7r" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252") : secret "image-registry-tls" not found Sep 10 18:32:11.626094 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:11.626065 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pb2pr" Sep 10 18:32:16.557575 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:16.557529 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5tq6l\" (UID: \"c6c62a12-04eb-40ce-b5cb-3a43fc562884\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:32:16.557978 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:16.557678 2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 18:32:16.557978 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:16.557746 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert podName:c6c62a12-04eb-40ce-b5cb-3a43fc562884 nodeName:}" failed. No retries permitted until 2026-09-10 18:32:48.557730294 +0000 UTC m=+97.701033516 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5tq6l" (UID: "c6c62a12-04eb-40ce-b5cb-3a43fc562884") : secret "networking-console-plugin-cert" not found Sep 10 18:32:16.657909 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:16.657871 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:32:16.658077 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:16.657937 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:32:16.658077 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:16.658038 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 18:32:16.658077 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:16.658038 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 18:32:16.658314 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:16.658117 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls podName:857e07ef-d71a-41f6-b863-b90126b0febb nodeName:}" failed. No retries permitted until 2026-09-10 18:32:48.658100622 +0000 UTC m=+97.801403843 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls") pod "dns-default-nggdz" (UID: "857e07ef-d71a-41f6-b863-b90126b0febb") : secret "dns-default-metrics-tls" not found Sep 10 18:32:16.658314 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:16.658049 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84ccb96897-cld7r: secret "image-registry-tls" not found Sep 10 18:32:16.658314 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:16.658171 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls podName:05dbe2c5-05cf-4b20-99e2-a50b46fe0252 nodeName:}" failed. No retries permitted until 2026-09-10 18:32:48.658158065 +0000 UTC m=+97.801461308 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls") pod "image-registry-84ccb96897-cld7r" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252") : secret "image-registry-tls" not found Sep 10 18:32:17.060389 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:17.060337 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret\") pod \"global-pull-secret-syncer-mm2nc\" (UID: \"25dec05c-c81f-4c62-a6e8-1d3506812ff5\") " pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:32:17.060389 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:17.060393 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs\") pod \"network-metrics-daemon-6dxjk\" (UID: \"99da651c-7b27-49ea-b49a-7f72caf21100\") " pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:32:17.062373 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:17.062352 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 18:32:17.062502 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:17.062410 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 18:32:17.071221 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:17.071204 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 18:32:17.071306 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:17.071282 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs podName:99da651c-7b27-49ea-b49a-7f72caf21100 nodeName:}" failed. No retries permitted until 2026-09-10 18:33:21.071260667 +0000 UTC m=+130.214563890 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs") pod "network-metrics-daemon-6dxjk" (UID: "99da651c-7b27-49ea-b49a-7f72caf21100") : secret "metrics-daemon-secret" not found Sep 10 18:32:17.073905 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:17.073879 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/25dec05c-c81f-4c62-a6e8-1d3506812ff5-original-pull-secret\") pod \"global-pull-secret-syncer-mm2nc\" (UID: \"25dec05c-c81f-4c62-a6e8-1d3506812ff5\") " pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:32:17.161542 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:17.161506 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gr9s\" (UniqueName: \"kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s\") pod \"network-check-target-dnfs8\" (UID: \"4f84d0cb-448c-4000-85cf-68e4e014c8ec\") " pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:32:17.163576 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:17.163560 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 18:32:17.173777 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:17.173758 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 18:32:17.184562 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:17.184539 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gr9s\" (UniqueName: \"kubernetes.io/projected/4f84d0cb-448c-4000-85cf-68e4e014c8ec-kube-api-access-8gr9s\") pod \"network-check-target-dnfs8\" (UID: \"4f84d0cb-448c-4000-85cf-68e4e014c8ec\") " pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:32:17.357817 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:17.357739 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dxnbp\"" Sep 10 18:32:17.364828 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:17.364805 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mm2nc" Sep 10 18:32:17.365892 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:17.365872 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:32:17.645091 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:17.645020 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dnfs8"] Sep 10 18:32:17.647856 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:17.647832 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mm2nc"] Sep 10 18:32:17.648892 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:32:17.648798 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f84d0cb_448c_4000_85cf_68e4e014c8ec.slice/crio-bf92b467c67f999dad05de1851d0fcb432ce512d78d835a9dccf0313a0523de9 WatchSource:0}: Error finding container bf92b467c67f999dad05de1851d0fcb432ce512d78d835a9dccf0313a0523de9: Status 404 returned error can't find the container with id bf92b467c67f999dad05de1851d0fcb432ce512d78d835a9dccf0313a0523de9 Sep 10 18:32:17.650445 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:32:17.650429 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25dec05c_c81f_4c62_a6e8_1d3506812ff5.slice/crio-265e87e2ec219a3d2038456cdf6e57839ee66e4beeb6b4f3f00e66c0cbabc341 WatchSource:0}: Error finding container 265e87e2ec219a3d2038456cdf6e57839ee66e4beeb6b4f3f00e66c0cbabc341: Status 404 returned error can't find the container with id 265e87e2ec219a3d2038456cdf6e57839ee66e4beeb6b4f3f00e66c0cbabc341 Sep 10 18:32:17.717543 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:17.717509 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dnfs8" event={"ID":"4f84d0cb-448c-4000-85cf-68e4e014c8ec","Type":"ContainerStarted","Data":"bf92b467c67f999dad05de1851d0fcb432ce512d78d835a9dccf0313a0523de9"} Sep 10 18:32:17.718551 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:17.718529 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mm2nc" event={"ID":"25dec05c-c81f-4c62-a6e8-1d3506812ff5","Type":"ContainerStarted","Data":"265e87e2ec219a3d2038456cdf6e57839ee66e4beeb6b4f3f00e66c0cbabc341"} Sep 10 18:32:23.735743 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:23.735707 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dnfs8" event={"ID":"4f84d0cb-448c-4000-85cf-68e4e014c8ec","Type":"ContainerStarted","Data":"69a7f3830d596c42d26d3a0ef85e1817c4eeff45028451f1678a22c874edf39f"} Sep 10 18:32:23.736219 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:23.735789 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:32:23.737541 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:23.737510 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mm2nc" event={"ID":"25dec05c-c81f-4c62-a6e8-1d3506812ff5","Type":"ContainerStarted","Data":"b51e0e0d98687a8d4627999df6cc8f1678bbc05e6bbcab1f38b91ac1590e4f8b"} Sep 10 18:32:23.752870 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:23.752823 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dnfs8" podStartSLOduration=66.850211115 podStartE2EDuration="1m12.752805855s" totalImagesPullingTime="5.90259474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:32:23.752033016 +0000 UTC m=+72.895336259" watchObservedRunningTime="2026-09-10 18:32:23.752805855 +0000 UTC m=+72.896109093" Sep 10 18:32:23.767048 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:23.767003 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mm2nc" podStartSLOduration=66.87566686 podStartE2EDuration="1m12.766988788s" totalImagesPullingTime="5.891321928s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:32:23.766198565 +0000 UTC m=+72.909501809" watchObservedRunningTime="2026-09-10 18:32:23.766988788 +0000 UTC m=+72.910292033" Sep 10 18:32:48.588011 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:48.587926 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5tq6l\" (UID: \"c6c62a12-04eb-40ce-b5cb-3a43fc562884\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:32:48.588340 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:48.588046 2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 18:32:48.588340 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:48.588099 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert podName:c6c62a12-04eb-40ce-b5cb-3a43fc562884 nodeName:}" failed. No retries permitted until 2026-09-10 18:33:52.588085454 +0000 UTC m=+161.731388676 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5tq6l" (UID: "c6c62a12-04eb-40ce-b5cb-3a43fc562884") : secret "networking-console-plugin-cert" not found Sep 10 18:32:48.688930 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:48.688879 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:32:48.689065 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:48.688970 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:32:48.689065 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:48.689012 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 18:32:48.689140 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:48.689077 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls podName:857e07ef-d71a-41f6-b863-b90126b0febb nodeName:}" failed. No retries permitted until 2026-09-10 18:33:52.689060473 +0000 UTC m=+161.832363719 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls") pod "dns-default-nggdz" (UID: "857e07ef-d71a-41f6-b863-b90126b0febb") : secret "dns-default-metrics-tls" not found Sep 10 18:32:48.689140 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:48.689084 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 18:32:48.689140 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:48.689095 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84ccb96897-cld7r: secret "image-registry-tls" not found Sep 10 18:32:48.689140 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:32:48.689136 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls podName:05dbe2c5-05cf-4b20-99e2-a50b46fe0252 nodeName:}" failed. No retries permitted until 2026-09-10 18:33:52.689120086 +0000 UTC m=+161.832423311 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls") pod "image-registry-84ccb96897-cld7r" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252") : secret "image-registry-tls" not found Sep 10 18:32:54.742543 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:32:54.742512 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dnfs8" Sep 10 18:33:21.122980 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:33:21.122941 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs\") pod \"network-metrics-daemon-6dxjk\" (UID: \"99da651c-7b27-49ea-b49a-7f72caf21100\") " pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:33:21.123360 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:33:21.123084 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 18:33:21.123360 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:33:21.123159 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs podName:99da651c-7b27-49ea-b49a-7f72caf21100 nodeName:}" failed. No retries permitted until 2026-09-10 18:35:23.123142997 +0000 UTC m=+252.266446220 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs") pod "network-metrics-daemon-6dxjk" (UID: "99da651c-7b27-49ea-b49a-7f72caf21100") : secret "metrics-daemon-secret" not found Sep 10 18:33:47.780034 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:33:47.779987 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" podUID="c6c62a12-04eb-40ce-b5cb-3a43fc562884" Sep 10 18:33:47.788168 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:33:47.788135 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" podUID="05dbe2c5-05cf-4b20-99e2-a50b46fe0252" Sep 10 18:33:47.803300 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:33:47.803274 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-nggdz" podUID="857e07ef-d71a-41f6-b863-b90126b0febb" Sep 10 18:33:47.942286 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:33:47.942258 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:33:47.942446 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:33:47.942261 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nggdz" Sep 10 18:33:47.942446 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:33:47.942261 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:33:49.469997 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:33:49.469961 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:33:52.642061 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:33:52.642002 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5tq6l\" (UID: \"c6c62a12-04eb-40ce-b5cb-3a43fc562884\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:33:52.642615 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:33:52.642174 2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 18:33:52.642615 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:33:52.642281 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert podName:c6c62a12-04eb-40ce-b5cb-3a43fc562884 nodeName:}" failed. No retries permitted until 2026-09-10 18:35:54.642257333 +0000 UTC m=+283.785560555 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-5tq6l" (UID: "c6c62a12-04eb-40ce-b5cb-3a43fc562884") : secret "networking-console-plugin-cert" not found Sep 10 18:33:52.742567 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:33:52.742523 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:33:52.742745 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:33:52.742601 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:33:52.742745 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:33:52.742682 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 18:33:52.742745 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:33:52.742742 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 18:33:52.742873 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:33:52.742757 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls podName:857e07ef-d71a-41f6-b863-b90126b0febb nodeName:}" failed. No retries permitted until 2026-09-10 18:35:54.742742889 +0000 UTC m=+283.886046134 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls") pod "dns-default-nggdz" (UID: "857e07ef-d71a-41f6-b863-b90126b0febb") : secret "dns-default-metrics-tls" not found Sep 10 18:33:52.742873 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:33:52.742757 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84ccb96897-cld7r: secret "image-registry-tls" not found Sep 10 18:33:52.742873 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:33:52.742786 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls podName:05dbe2c5-05cf-4b20-99e2-a50b46fe0252 nodeName:}" failed. No retries permitted until 2026-09-10 18:35:54.742780215 +0000 UTC m=+283.886083483 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls") pod "image-registry-84ccb96897-cld7r" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252") : secret "image-registry-tls" not found Sep 10 18:34:01.442939 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:34:01.442899 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:35:08.433756 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.433716 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ggp6z"] Sep 10 18:35:08.436522 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.436479 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ggp6z" Sep 10 18:35:08.437297 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.437268 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9"] Sep 10 18:35:08.438764 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.438743 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-p47cv\"" Sep 10 18:35:08.439105 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.439030 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 10 18:35:08.439370 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.439354 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 10 18:35:08.440182 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.440165 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-d28xz"] Sep 10 18:35:08.440299 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.440279 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9" Sep 10 18:35:08.441936 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.441920 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 10 18:35:08.442099 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.442079 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 10 18:35:08.442262 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.442242 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-9fft6\"" Sep 10 18:35:08.442324 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.442275 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 10 18:35:08.442443 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.442424 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 10 18:35:08.442899 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.442884 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:08.444322 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.444305 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 10 18:35:08.444952 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.444930 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 10 18:35:08.444952 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.444938 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 10 18:35:08.445109 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.444954 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 10 18:35:08.445109 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.445012 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-lfgpz\"" Sep 10 18:35:08.448845 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.448827 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ggp6z"] Sep 10 18:35:08.449857 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.449833 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9"] Sep 10 18:35:08.451021 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.450945 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 10 18:35:08.462464 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.462444 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-d28xz"] Sep 10 18:35:08.530116 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.530079 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx"] Sep 10 18:35:08.532945 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.532923 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-4m5dh"] Sep 10 18:35:08.533060 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.533036 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx" Sep 10 18:35:08.535407 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.535382 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 10 18:35:08.535677 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.535660 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-mk8p4\"" Sep 10 18:35:08.535908 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.535893 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 10 18:35:08.536163 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.536141 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 10 18:35:08.536520 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.536502 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 10 18:35:08.537074 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.537052 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-4m5dh" Sep 10 18:35:08.539183 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.539165 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qn4fv\"" Sep 10 18:35:08.541970 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.541947 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx"] Sep 10 18:35:08.545309 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.545290 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-4m5dh"] Sep 10 18:35:08.553164 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.553145 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3c15c29b-edff-4386-a50a-177b41560acd-config\") pod \"console-operator-85b454874b-d28xz\" (UID: \"3c15c29b-edff-4386-a50a-177b41560acd\") " pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:08.553254 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.553169 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3c15c29b-edff-4386-a50a-177b41560acd-serving-cert\") pod \"console-operator-85b454874b-d28xz\" (UID: \"3c15c29b-edff-4386-a50a-177b41560acd\") " pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:08.553254 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.553188 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c15c29b-edff-4386-a50a-177b41560acd-trusted-ca\") pod \"console-operator-85b454874b-d28xz\" (UID: \"3c15c29b-edff-4386-a50a-177b41560acd\") " pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:08.553254 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.553210 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/418a0ba4-c771-43aa-bc0c-baf2b0b4f34f-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-sm6d9\" (UID: \"418a0ba4-c771-43aa-bc0c-baf2b0b4f34f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9" Sep 10 18:35:08.553384 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.553317 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ppbc\" (UniqueName: \"kubernetes.io/projected/418a0ba4-c771-43aa-bc0c-baf2b0b4f34f-kube-api-access-5ppbc\") pod \"kube-storage-version-migrator-operator-8d7dd4675-sm6d9\" (UID: \"418a0ba4-c771-43aa-bc0c-baf2b0b4f34f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9" Sep 10 18:35:08.553384 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.553357 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/418a0ba4-c771-43aa-bc0c-baf2b0b4f34f-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-sm6d9\" (UID: \"418a0ba4-c771-43aa-bc0c-baf2b0b4f34f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9" Sep 10 18:35:08.553509 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.553435 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvjjg\" (UniqueName: \"kubernetes.io/projected/3c15c29b-edff-4386-a50a-177b41560acd-kube-api-access-xvjjg\") pod \"console-operator-85b454874b-d28xz\" (UID: \"3c15c29b-edff-4386-a50a-177b41560acd\") " pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:08.553509 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.553495 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkzkj\" (UniqueName: \"kubernetes.io/projected/5da5cd5f-ac01-4cf5-868b-ed1f600ef8d5-kube-api-access-tkzkj\") pod \"volume-data-source-validator-695c895bd6-ggp6z\" (UID: \"5da5cd5f-ac01-4cf5-868b-ed1f600ef8d5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ggp6z" Sep 10 18:35:08.654336 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.654298 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2qc6\" (UniqueName: \"kubernetes.io/projected/e0532948-ea6e-452e-b5c6-092e5d06e02c-kube-api-access-v2qc6\") pod \"network-check-source-685456655f-4m5dh\" (UID: \"e0532948-ea6e-452e-b5c6-092e5d06e02c\") " pod="openshift-network-diagnostics/network-check-source-685456655f-4m5dh" Sep 10 18:35:08.654336 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.654337 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89753db4-e14c-4c10-9b8a-773a684985e3-serving-cert\") pod \"service-ca-operator-859f94c4b-rzdsx\" (UID: \"89753db4-e14c-4c10-9b8a-773a684985e3\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx" Sep 10 18:35:08.654627 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.654361 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3c15c29b-edff-4386-a50a-177b41560acd-config\") pod \"console-operator-85b454874b-d28xz\" (UID: \"3c15c29b-edff-4386-a50a-177b41560acd\") " pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:08.654627 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.654377 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3c15c29b-edff-4386-a50a-177b41560acd-serving-cert\") pod \"console-operator-85b454874b-d28xz\" (UID: \"3c15c29b-edff-4386-a50a-177b41560acd\") " pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:08.654627 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.654393 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/89753db4-e14c-4c10-9b8a-773a684985e3-config\") pod \"service-ca-operator-859f94c4b-rzdsx\" (UID: \"89753db4-e14c-4c10-9b8a-773a684985e3\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx" Sep 10 18:35:08.654627 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.654412 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c15c29b-edff-4386-a50a-177b41560acd-trusted-ca\") pod \"console-operator-85b454874b-d28xz\" (UID: \"3c15c29b-edff-4386-a50a-177b41560acd\") " pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:08.654627 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.654507 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/418a0ba4-c771-43aa-bc0c-baf2b0b4f34f-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-sm6d9\" (UID: \"418a0ba4-c771-43aa-bc0c-baf2b0b4f34f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9" Sep 10 18:35:08.654627 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.654548 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ppbc\" (UniqueName: \"kubernetes.io/projected/418a0ba4-c771-43aa-bc0c-baf2b0b4f34f-kube-api-access-5ppbc\") pod \"kube-storage-version-migrator-operator-8d7dd4675-sm6d9\" (UID: \"418a0ba4-c771-43aa-bc0c-baf2b0b4f34f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9" Sep 10 18:35:08.654627 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.654568 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/418a0ba4-c771-43aa-bc0c-baf2b0b4f34f-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-sm6d9\" (UID: \"418a0ba4-c771-43aa-bc0c-baf2b0b4f34f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9" Sep 10 18:35:08.654969 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.654715 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvjjg\" (UniqueName: \"kubernetes.io/projected/3c15c29b-edff-4386-a50a-177b41560acd-kube-api-access-xvjjg\") pod \"console-operator-85b454874b-d28xz\" (UID: \"3c15c29b-edff-4386-a50a-177b41560acd\") " pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:08.654969 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.654759 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkzkj\" (UniqueName: \"kubernetes.io/projected/5da5cd5f-ac01-4cf5-868b-ed1f600ef8d5-kube-api-access-tkzkj\") pod \"volume-data-source-validator-695c895bd6-ggp6z\" (UID: \"5da5cd5f-ac01-4cf5-868b-ed1f600ef8d5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ggp6z" Sep 10 18:35:08.654969 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.654789 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4jpl\" (UniqueName: \"kubernetes.io/projected/89753db4-e14c-4c10-9b8a-773a684985e3-kube-api-access-r4jpl\") pod \"service-ca-operator-859f94c4b-rzdsx\" (UID: \"89753db4-e14c-4c10-9b8a-773a684985e3\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx" Sep 10 18:35:08.655118 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.655080 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3c15c29b-edff-4386-a50a-177b41560acd-config\") pod \"console-operator-85b454874b-d28xz\" (UID: \"3c15c29b-edff-4386-a50a-177b41560acd\") " pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:08.655453 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.655428 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c15c29b-edff-4386-a50a-177b41560acd-trusted-ca\") pod \"console-operator-85b454874b-d28xz\" (UID: \"3c15c29b-edff-4386-a50a-177b41560acd\") " pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:08.655568 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.655542 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/418a0ba4-c771-43aa-bc0c-baf2b0b4f34f-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-sm6d9\" (UID: \"418a0ba4-c771-43aa-bc0c-baf2b0b4f34f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9" Sep 10 18:35:08.656768 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.656746 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/418a0ba4-c771-43aa-bc0c-baf2b0b4f34f-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-sm6d9\" (UID: \"418a0ba4-c771-43aa-bc0c-baf2b0b4f34f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9" Sep 10 18:35:08.656878 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.656860 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3c15c29b-edff-4386-a50a-177b41560acd-serving-cert\") pod \"console-operator-85b454874b-d28xz\" (UID: \"3c15c29b-edff-4386-a50a-177b41560acd\") " pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:08.661755 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.661737 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ppbc\" (UniqueName: \"kubernetes.io/projected/418a0ba4-c771-43aa-bc0c-baf2b0b4f34f-kube-api-access-5ppbc\") pod \"kube-storage-version-migrator-operator-8d7dd4675-sm6d9\" (UID: \"418a0ba4-c771-43aa-bc0c-baf2b0b4f34f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9" Sep 10 18:35:08.661854 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.661776 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvjjg\" (UniqueName: \"kubernetes.io/projected/3c15c29b-edff-4386-a50a-177b41560acd-kube-api-access-xvjjg\") pod \"console-operator-85b454874b-d28xz\" (UID: \"3c15c29b-edff-4386-a50a-177b41560acd\") " pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:08.661955 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.661937 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkzkj\" (UniqueName: \"kubernetes.io/projected/5da5cd5f-ac01-4cf5-868b-ed1f600ef8d5-kube-api-access-tkzkj\") pod \"volume-data-source-validator-695c895bd6-ggp6z\" (UID: \"5da5cd5f-ac01-4cf5-868b-ed1f600ef8d5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ggp6z" Sep 10 18:35:08.747929 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.747897 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ggp6z" Sep 10 18:35:08.755207 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.755185 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9" Sep 10 18:35:08.755365 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.755345 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4jpl\" (UniqueName: \"kubernetes.io/projected/89753db4-e14c-4c10-9b8a-773a684985e3-kube-api-access-r4jpl\") pod \"service-ca-operator-859f94c4b-rzdsx\" (UID: \"89753db4-e14c-4c10-9b8a-773a684985e3\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx" Sep 10 18:35:08.755419 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.755408 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2qc6\" (UniqueName: \"kubernetes.io/projected/e0532948-ea6e-452e-b5c6-092e5d06e02c-kube-api-access-v2qc6\") pod \"network-check-source-685456655f-4m5dh\" (UID: \"e0532948-ea6e-452e-b5c6-092e5d06e02c\") " pod="openshift-network-diagnostics/network-check-source-685456655f-4m5dh" Sep 10 18:35:08.755455 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.755427 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89753db4-e14c-4c10-9b8a-773a684985e3-serving-cert\") pod \"service-ca-operator-859f94c4b-rzdsx\" (UID: \"89753db4-e14c-4c10-9b8a-773a684985e3\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx" Sep 10 18:35:08.755455 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.755445 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/89753db4-e14c-4c10-9b8a-773a684985e3-config\") pod \"service-ca-operator-859f94c4b-rzdsx\" (UID: \"89753db4-e14c-4c10-9b8a-773a684985e3\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx" Sep 10 18:35:08.756026 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.756000 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/89753db4-e14c-4c10-9b8a-773a684985e3-config\") pod \"service-ca-operator-859f94c4b-rzdsx\" (UID: \"89753db4-e14c-4c10-9b8a-773a684985e3\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx" Sep 10 18:35:08.757897 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.757866 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89753db4-e14c-4c10-9b8a-773a684985e3-serving-cert\") pod \"service-ca-operator-859f94c4b-rzdsx\" (UID: \"89753db4-e14c-4c10-9b8a-773a684985e3\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx" Sep 10 18:35:08.760452 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.760429 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:08.762670 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.762645 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2qc6\" (UniqueName: \"kubernetes.io/projected/e0532948-ea6e-452e-b5c6-092e5d06e02c-kube-api-access-v2qc6\") pod \"network-check-source-685456655f-4m5dh\" (UID: \"e0532948-ea6e-452e-b5c6-092e5d06e02c\") " pod="openshift-network-diagnostics/network-check-source-685456655f-4m5dh" Sep 10 18:35:08.763098 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.763075 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4jpl\" (UniqueName: \"kubernetes.io/projected/89753db4-e14c-4c10-9b8a-773a684985e3-kube-api-access-r4jpl\") pod \"service-ca-operator-859f94c4b-rzdsx\" (UID: \"89753db4-e14c-4c10-9b8a-773a684985e3\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx" Sep 10 18:35:08.844692 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.844663 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx" Sep 10 18:35:08.850630 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:08.850608 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-4m5dh" Sep 10 18:35:09.023154 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:09.023085 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ggp6z"] Sep 10 18:35:09.026089 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:35:09.026058 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da5cd5f_ac01_4cf5_868b_ed1f600ef8d5.slice/crio-4d728a68faaffbc059a432432a4e6a347564ee94a7c3d1dfaf04452e3a77026d WatchSource:0}: Error finding container 4d728a68faaffbc059a432432a4e6a347564ee94a7c3d1dfaf04452e3a77026d: Status 404 returned error can't find the container with id 4d728a68faaffbc059a432432a4e6a347564ee94a7c3d1dfaf04452e3a77026d Sep 10 18:35:09.027327 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:09.027302 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9"] Sep 10 18:35:09.030384 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:35:09.030358 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod418a0ba4_c771_43aa_bc0c_baf2b0b4f34f.slice/crio-ccb9dedb28e3859ae988d3b2af96e2224f230ef0062d4ebfaf98e0d60c6d2054 WatchSource:0}: Error finding container ccb9dedb28e3859ae988d3b2af96e2224f230ef0062d4ebfaf98e0d60c6d2054: Status 404 returned error can't find the container with id ccb9dedb28e3859ae988d3b2af96e2224f230ef0062d4ebfaf98e0d60c6d2054 Sep 10 18:35:09.046975 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:09.046940 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-d28xz"] Sep 10 18:35:09.050539 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:35:09.050519 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c15c29b_edff_4386_a50a_177b41560acd.slice/crio-eed6ad6a92d2e7b6679caaa5f0f6b65e18587c1ae3f0ef0e1f304db652456d20 WatchSource:0}: Error finding container eed6ad6a92d2e7b6679caaa5f0f6b65e18587c1ae3f0ef0e1f304db652456d20: Status 404 returned error can't find the container with id eed6ad6a92d2e7b6679caaa5f0f6b65e18587c1ae3f0ef0e1f304db652456d20 Sep 10 18:35:09.119757 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:09.119725 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx"] Sep 10 18:35:09.123896 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:09.123866 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-4m5dh"] Sep 10 18:35:09.124555 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:35:09.124525 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89753db4_e14c_4c10_9b8a_773a684985e3.slice/crio-cbdb6b30018fbce7236436169a5533b9567023afe1800f91becae0f1f0868cf9 WatchSource:0}: Error finding container cbdb6b30018fbce7236436169a5533b9567023afe1800f91becae0f1f0868cf9: Status 404 returned error can't find the container with id cbdb6b30018fbce7236436169a5533b9567023afe1800f91becae0f1f0868cf9 Sep 10 18:35:09.128914 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:35:09.128890 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0532948_ea6e_452e_b5c6_092e5d06e02c.slice/crio-8d3e3da11b2e1dca4fb4abf396d468ff2e74dbd68285063d83da7ef7175fc774 WatchSource:0}: Error finding container 8d3e3da11b2e1dca4fb4abf396d468ff2e74dbd68285063d83da7ef7175fc774: Status 404 returned error can't find the container with id 8d3e3da11b2e1dca4fb4abf396d468ff2e74dbd68285063d83da7ef7175fc774 Sep 10 18:35:09.137459 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:09.137438 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-d28xz" event={"ID":"3c15c29b-edff-4386-a50a-177b41560acd","Type":"ContainerStarted","Data":"eed6ad6a92d2e7b6679caaa5f0f6b65e18587c1ae3f0ef0e1f304db652456d20"} Sep 10 18:35:09.138598 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:09.138579 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9" event={"ID":"418a0ba4-c771-43aa-bc0c-baf2b0b4f34f","Type":"ContainerStarted","Data":"ccb9dedb28e3859ae988d3b2af96e2224f230ef0062d4ebfaf98e0d60c6d2054"} Sep 10 18:35:09.139884 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:09.139861 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ggp6z" event={"ID":"5da5cd5f-ac01-4cf5-868b-ed1f600ef8d5","Type":"ContainerStarted","Data":"4d728a68faaffbc059a432432a4e6a347564ee94a7c3d1dfaf04452e3a77026d"} Sep 10 18:35:09.140938 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:09.140919 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-4m5dh" event={"ID":"e0532948-ea6e-452e-b5c6-092e5d06e02c","Type":"ContainerStarted","Data":"8d3e3da11b2e1dca4fb4abf396d468ff2e74dbd68285063d83da7ef7175fc774"} Sep 10 18:35:09.142086 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:09.142064 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx" event={"ID":"89753db4-e14c-4c10-9b8a-773a684985e3","Type":"ContainerStarted","Data":"cbdb6b30018fbce7236436169a5533b9567023afe1800f91becae0f1f0868cf9"} Sep 10 18:35:10.150289 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:10.149624 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-4m5dh" event={"ID":"e0532948-ea6e-452e-b5c6-092e5d06e02c","Type":"ContainerStarted","Data":"710f2a3f51ecfe15c1ccd2d7ccd05e7ed3b1e78b616a1a9de20d50f5dced00f7"} Sep 10 18:35:10.164249 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:10.164190 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-4m5dh" podStartSLOduration=2.164169333 podStartE2EDuration="2.164169333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:35:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:35:10.162753012 +0000 UTC m=+239.306056257" watchObservedRunningTime="2026-09-10 18:35:10.164169333 +0000 UTC m=+239.307472582" Sep 10 18:35:11.155516 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:11.155462 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ggp6z" event={"ID":"5da5cd5f-ac01-4cf5-868b-ed1f600ef8d5","Type":"ContainerStarted","Data":"d941a5d3389a5f70a2204711d4b5117104538af9a6466b0f5407c44689a06b42"} Sep 10 18:35:11.169746 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:11.169692 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-ggp6z" podStartSLOduration=1.723620129 podStartE2EDuration="3.169672708s" totalImagesPullingTime="1.446052579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:35:11.168467081 +0000 UTC m=+240.311770318" watchObservedRunningTime="2026-09-10 18:35:11.169672708 +0000 UTC m=+240.312975955" Sep 10 18:35:12.728069 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.728035 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jxt56"] Sep 10 18:35:12.731060 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.731044 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:12.734325 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.734301 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 18:35:12.734436 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.734400 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 18:35:12.734436 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.734405 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 18:35:12.734813 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.734793 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n67nc\"" Sep 10 18:35:12.742781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.742765 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 18:35:12.751011 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.750989 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jxt56"] Sep 10 18:35:12.791459 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.791432 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9eb589d3-13f2-4138-948a-373b4e6b8dff-crio-socket\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:12.791599 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.791475 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9eb589d3-13f2-4138-948a-373b4e6b8dff-data-volume\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:12.791599 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.791507 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxwh4\" (UniqueName: \"kubernetes.io/projected/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-api-access-bxwh4\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:12.791749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.791646 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:12.791749 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.791683 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:12.892544 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.892512 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9eb589d3-13f2-4138-948a-373b4e6b8dff-crio-socket\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:12.892695 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.892559 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9eb589d3-13f2-4138-948a-373b4e6b8dff-data-volume\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:12.892695 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.892579 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxwh4\" (UniqueName: \"kubernetes.io/projected/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-api-access-bxwh4\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:12.892695 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.892621 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:12.892695 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.892637 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:12.892695 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.892649 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9eb589d3-13f2-4138-948a-373b4e6b8dff-crio-socket\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:12.892957 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:12.892738 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 18:35:12.892957 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:12.892758 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 18:35:12.892957 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:12.892816 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm podName:9eb589d3-13f2-4138-948a-373b4e6b8dff nodeName:}" failed. No retries permitted until 2026-09-10 18:35:13.392794849 +0000 UTC m=+242.536098070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxt56" (UID: "9eb589d3-13f2-4138-948a-373b4e6b8dff") : configmap "kube-rbac-proxy" not found Sep 10 18:35:12.892957 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:12.892836 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls podName:9eb589d3-13f2-4138-948a-373b4e6b8dff nodeName:}" failed. No retries permitted until 2026-09-10 18:35:13.392825969 +0000 UTC m=+242.536129206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxt56" (UID: "9eb589d3-13f2-4138-948a-373b4e6b8dff") : secret "insights-runtime-extractor-tls" not found Sep 10 18:35:12.892957 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.892894 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9eb589d3-13f2-4138-948a-373b4e6b8dff-data-volume\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:12.900313 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:12.900287 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxwh4\" (UniqueName: \"kubernetes.io/projected/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-api-access-bxwh4\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:13.164978 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.164899 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d28xz_3c15c29b-edff-4386-a50a-177b41560acd/console-operator/0.log" Sep 10 18:35:13.164978 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.164947 2564 generic.go:356] "Generic (PLEG): container finished" podID="3c15c29b-edff-4386-a50a-177b41560acd" containerID="0c057726ab75ecf3da06d60085b5a7fdda8418c6a0d1b8f04ceca11c4c7c9a7b" exitCode=255 Sep 10 18:35:13.165203 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.165034 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-d28xz" event={"ID":"3c15c29b-edff-4386-a50a-177b41560acd","Type":"ContainerDied","Data":"0c057726ab75ecf3da06d60085b5a7fdda8418c6a0d1b8f04ceca11c4c7c9a7b"} Sep 10 18:35:13.165278 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.165233 2564 scope.go:117] "RemoveContainer" containerID="0c057726ab75ecf3da06d60085b5a7fdda8418c6a0d1b8f04ceca11c4c7c9a7b" Sep 10 18:35:13.166720 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.166698 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9" event={"ID":"418a0ba4-c771-43aa-bc0c-baf2b0b4f34f","Type":"ContainerStarted","Data":"61ce64b9e82adc1a8c372dbdf0dea0f1668be94f96718fee50e3667673417b93"} Sep 10 18:35:13.171055 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.171033 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx" event={"ID":"89753db4-e14c-4c10-9b8a-773a684985e3","Type":"ContainerStarted","Data":"985e50f8506ae63bddf960194a10f4d81417ecacf3ac0ebc4cf5d3460a879966"} Sep 10 18:35:13.196181 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.196141 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9" podStartSLOduration=1.873923044 podStartE2EDuration="5.196126808s" totalImagesPullingTime="3.322203764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:35:13.195781499 +0000 UTC m=+242.339084727" watchObservedRunningTime="2026-09-10 18:35:13.196126808 +0000 UTC m=+242.339430050" Sep 10 18:35:13.397125 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.397093 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:13.397295 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.397130 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:13.397295 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:13.397208 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 18:35:13.397295 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:13.397261 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 18:35:13.397295 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:13.397274 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm podName:9eb589d3-13f2-4138-948a-373b4e6b8dff nodeName:}" failed. No retries permitted until 2026-09-10 18:35:14.397254359 +0000 UTC m=+243.540557599 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxt56" (UID: "9eb589d3-13f2-4138-948a-373b4e6b8dff") : configmap "kube-rbac-proxy" not found Sep 10 18:35:13.397295 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:13.397295 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls podName:9eb589d3-13f2-4138-948a-373b4e6b8dff nodeName:}" failed. No retries permitted until 2026-09-10 18:35:14.397284955 +0000 UTC m=+243.540588194 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxt56" (UID: "9eb589d3-13f2-4138-948a-373b4e6b8dff") : secret "insights-runtime-extractor-tls" not found Sep 10 18:35:13.822908 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.822856 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx" podStartSLOduration=2.5915492 podStartE2EDuration="5.822823265s" totalImagesPullingTime="3.231274065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:35:13.209439178 +0000 UTC m=+242.352742425" watchObservedRunningTime="2026-09-10 18:35:13.822823265 +0000 UTC m=+242.966126531" Sep 10 18:35:13.823513 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.822991 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-s6687"] Sep 10 18:35:13.825828 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.825804 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-s6687" Sep 10 18:35:13.828058 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.828030 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-8skx9\"" Sep 10 18:35:13.828334 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.828311 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 10 18:35:13.828509 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.828458 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 10 18:35:13.835308 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.835282 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-s6687"] Sep 10 18:35:13.901677 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:13.901641 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f7t4\" (UniqueName: \"kubernetes.io/projected/eba65242-6e91-4208-91db-6f590e66aebc-kube-api-access-4f7t4\") pod \"migrator-fdbb8fdff-s6687\" (UID: \"eba65242-6e91-4208-91db-6f590e66aebc\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-s6687" Sep 10 18:35:14.003029 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:14.002983 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f7t4\" (UniqueName: \"kubernetes.io/projected/eba65242-6e91-4208-91db-6f590e66aebc-kube-api-access-4f7t4\") pod \"migrator-fdbb8fdff-s6687\" (UID: \"eba65242-6e91-4208-91db-6f590e66aebc\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-s6687" Sep 10 18:35:14.010251 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:14.010214 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f7t4\" (UniqueName: \"kubernetes.io/projected/eba65242-6e91-4208-91db-6f590e66aebc-kube-api-access-4f7t4\") pod \"migrator-fdbb8fdff-s6687\" (UID: \"eba65242-6e91-4208-91db-6f590e66aebc\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-s6687" Sep 10 18:35:14.137099 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:14.137028 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-s6687" Sep 10 18:35:14.176997 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:14.176973 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d28xz_3c15c29b-edff-4386-a50a-177b41560acd/console-operator/1.log" Sep 10 18:35:14.177394 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:14.177376 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d28xz_3c15c29b-edff-4386-a50a-177b41560acd/console-operator/0.log" Sep 10 18:35:14.177510 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:14.177416 2564 generic.go:356] "Generic (PLEG): container finished" podID="3c15c29b-edff-4386-a50a-177b41560acd" containerID="51ff3182d6c39685ac5bef2b3ac5328886faf61754c2f3ceb57d6ae645dec347" exitCode=255 Sep 10 18:35:14.177567 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:14.177519 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-d28xz" event={"ID":"3c15c29b-edff-4386-a50a-177b41560acd","Type":"ContainerDied","Data":"51ff3182d6c39685ac5bef2b3ac5328886faf61754c2f3ceb57d6ae645dec347"} Sep 10 18:35:14.177615 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:14.177575 2564 scope.go:117] "RemoveContainer" containerID="0c057726ab75ecf3da06d60085b5a7fdda8418c6a0d1b8f04ceca11c4c7c9a7b" Sep 10 18:35:14.177807 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:14.177783 2564 scope.go:117] "RemoveContainer" containerID="51ff3182d6c39685ac5bef2b3ac5328886faf61754c2f3ceb57d6ae645dec347" Sep 10 18:35:14.178018 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:14.177994 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-d28xz_openshift-console-operator(3c15c29b-edff-4386-a50a-177b41560acd)\"" pod="openshift-console-operator/console-operator-85b454874b-d28xz" podUID="3c15c29b-edff-4386-a50a-177b41560acd" Sep 10 18:35:14.404456 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:14.404381 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-s6687"] Sep 10 18:35:14.406525 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:14.406497 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:14.406674 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:14.406536 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:14.406674 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:14.406581 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 18:35:14.406674 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:14.406658 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm podName:9eb589d3-13f2-4138-948a-373b4e6b8dff nodeName:}" failed. No retries permitted until 2026-09-10 18:35:16.406638592 +0000 UTC m=+245.549941833 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxt56" (UID: "9eb589d3-13f2-4138-948a-373b4e6b8dff") : configmap "kube-rbac-proxy" not found Sep 10 18:35:14.406674 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:14.406671 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 18:35:14.406886 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:14.406726 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls podName:9eb589d3-13f2-4138-948a-373b4e6b8dff nodeName:}" failed. No retries permitted until 2026-09-10 18:35:16.406709193 +0000 UTC m=+245.550012417 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxt56" (UID: "9eb589d3-13f2-4138-948a-373b4e6b8dff") : secret "insights-runtime-extractor-tls" not found Sep 10 18:35:14.407096 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:35:14.407075 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeba65242_6e91_4208_91db_6f590e66aebc.slice/crio-040876c2ce0bc285069055abee68493c95470277dc4aa7f8481f761d4898d77d WatchSource:0}: Error finding container 040876c2ce0bc285069055abee68493c95470277dc4aa7f8481f761d4898d77d: Status 404 returned error can't find the container with id 040876c2ce0bc285069055abee68493c95470277dc4aa7f8481f761d4898d77d Sep 10 18:35:14.483455 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:14.483430 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4lr7x_e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0/dns-node-resolver/0.log" Sep 10 18:35:15.182901 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:15.182875 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d28xz_3c15c29b-edff-4386-a50a-177b41560acd/console-operator/1.log" Sep 10 18:35:15.183372 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:15.183268 2564 scope.go:117] "RemoveContainer" containerID="51ff3182d6c39685ac5bef2b3ac5328886faf61754c2f3ceb57d6ae645dec347" Sep 10 18:35:15.183477 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:15.183452 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-d28xz_openshift-console-operator(3c15c29b-edff-4386-a50a-177b41560acd)\"" pod="openshift-console-operator/console-operator-85b454874b-d28xz" podUID="3c15c29b-edff-4386-a50a-177b41560acd" Sep 10 18:35:15.184370 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:15.184340 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-s6687" event={"ID":"eba65242-6e91-4208-91db-6f590e66aebc","Type":"ContainerStarted","Data":"040876c2ce0bc285069055abee68493c95470277dc4aa7f8481f761d4898d77d"} Sep 10 18:35:15.483253 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:15.483228 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cd2ls_96332d76-e827-4612-a6b5-bd9d31fe347e/node-ca/0.log" Sep 10 18:35:16.189231 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:16.189147 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-s6687" event={"ID":"eba65242-6e91-4208-91db-6f590e66aebc","Type":"ContainerStarted","Data":"22e14de88efeb50f4b558f1d8442874e705f1bb2b2c177d990ea4ffbbe6d1b77"} Sep 10 18:35:16.189231 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:16.189216 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-s6687" event={"ID":"eba65242-6e91-4208-91db-6f590e66aebc","Type":"ContainerStarted","Data":"ceb65adc07c38ebc2ec48c645934da9ec3aa138db16cc0a0b08ef344cb68bb78"} Sep 10 18:35:16.202614 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:16.202564 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-s6687" podStartSLOduration=1.844028268 podStartE2EDuration="3.202549158s" totalImagesPullingTime="1.35852089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:35:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:35:16.202021189 +0000 UTC m=+245.345324486" watchObservedRunningTime="2026-09-10 18:35:16.202549158 +0000 UTC m=+245.345852402" Sep 10 18:35:16.424031 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:16.423990 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:16.424209 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:16.424036 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:16.424209 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:16.424111 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 18:35:16.424209 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:16.424173 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm podName:9eb589d3-13f2-4138-948a-373b4e6b8dff nodeName:}" failed. No retries permitted until 2026-09-10 18:35:20.424159113 +0000 UTC m=+249.567462335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxt56" (UID: "9eb589d3-13f2-4138-948a-373b4e6b8dff") : configmap "kube-rbac-proxy" not found Sep 10 18:35:16.424348 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:16.424216 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 18:35:16.424348 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:16.424271 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls podName:9eb589d3-13f2-4138-948a-373b4e6b8dff nodeName:}" failed. No retries permitted until 2026-09-10 18:35:20.424258386 +0000 UTC m=+249.567561608 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxt56" (UID: "9eb589d3-13f2-4138-948a-373b4e6b8dff") : secret "insights-runtime-extractor-tls" not found Sep 10 18:35:16.688711 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:16.688679 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zkgwb_cc9720ca-244c-4b07-88e5-e95fbeae172d/serve-healthcheck-canary/0.log" Sep 10 18:35:18.761753 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:18.761717 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:18.762106 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:18.761776 2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:18.762106 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:18.762075 2564 scope.go:117] "RemoveContainer" containerID="51ff3182d6c39685ac5bef2b3ac5328886faf61754c2f3ceb57d6ae645dec347" Sep 10 18:35:18.762229 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:18.762210 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-d28xz_openshift-console-operator(3c15c29b-edff-4386-a50a-177b41560acd)\"" pod="openshift-console-operator/console-operator-85b454874b-d28xz" podUID="3c15c29b-edff-4386-a50a-177b41560acd" Sep 10 18:35:20.455959 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:20.455923 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:20.455959 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:20.455959 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:20.456420 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:20.456044 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 18:35:20.456420 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:20.456070 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 18:35:20.456420 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:20.456106 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls podName:9eb589d3-13f2-4138-948a-373b4e6b8dff nodeName:}" failed. No retries permitted until 2026-09-10 18:35:28.456093019 +0000 UTC m=+257.599396254 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxt56" (UID: "9eb589d3-13f2-4138-948a-373b4e6b8dff") : secret "insights-runtime-extractor-tls" not found Sep 10 18:35:20.456420 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:20.456153 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm podName:9eb589d3-13f2-4138-948a-373b4e6b8dff nodeName:}" failed. No retries permitted until 2026-09-10 18:35:28.456134546 +0000 UTC m=+257.599437768 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxt56" (UID: "9eb589d3-13f2-4138-948a-373b4e6b8dff") : configmap "kube-rbac-proxy" not found Sep 10 18:35:23.180184 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:23.180145 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs\") pod \"network-metrics-daemon-6dxjk\" (UID: \"99da651c-7b27-49ea-b49a-7f72caf21100\") " pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:35:23.180598 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:23.180290 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 18:35:23.180598 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:23.180355 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs podName:99da651c-7b27-49ea-b49a-7f72caf21100 nodeName:}" failed. No retries permitted until 2026-09-10 18:37:25.180338532 +0000 UTC m=+374.323641755 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs") pod "network-metrics-daemon-6dxjk" (UID: "99da651c-7b27-49ea-b49a-7f72caf21100") : secret "metrics-daemon-secret" not found Sep 10 18:35:28.526272 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:28.526219 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:28.526272 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:28.526275 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:28.526745 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:28.526403 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 18:35:28.526745 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:28.526460 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls podName:9eb589d3-13f2-4138-948a-373b4e6b8dff nodeName:}" failed. No retries permitted until 2026-09-10 18:35:44.526444201 +0000 UTC m=+273.669747444 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxt56" (UID: "9eb589d3-13f2-4138-948a-373b4e6b8dff") : secret "insights-runtime-extractor-tls" not found Sep 10 18:35:28.526816 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:28.526769 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9eb589d3-13f2-4138-948a-373b4e6b8dff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:31.442950 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:31.442921 2564 scope.go:117] "RemoveContainer" containerID="51ff3182d6c39685ac5bef2b3ac5328886faf61754c2f3ceb57d6ae645dec347" Sep 10 18:35:32.239832 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:32.239800 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d28xz_3c15c29b-edff-4386-a50a-177b41560acd/console-operator/2.log" Sep 10 18:35:32.240163 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:32.240150 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d28xz_3c15c29b-edff-4386-a50a-177b41560acd/console-operator/1.log" Sep 10 18:35:32.240209 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:32.240183 2564 generic.go:356] "Generic (PLEG): container finished" podID="3c15c29b-edff-4386-a50a-177b41560acd" containerID="8b13450c47aa192772f84e4ecb69b7d2261643dd795bcd78bb0881fedd52722e" exitCode=255 Sep 10 18:35:32.240248 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:32.240221 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-d28xz" event={"ID":"3c15c29b-edff-4386-a50a-177b41560acd","Type":"ContainerDied","Data":"8b13450c47aa192772f84e4ecb69b7d2261643dd795bcd78bb0881fedd52722e"} Sep 10 18:35:32.240310 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:32.240296 2564 scope.go:117] "RemoveContainer" containerID="51ff3182d6c39685ac5bef2b3ac5328886faf61754c2f3ceb57d6ae645dec347" Sep 10 18:35:32.240605 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:32.240585 2564 scope.go:117] "RemoveContainer" containerID="8b13450c47aa192772f84e4ecb69b7d2261643dd795bcd78bb0881fedd52722e" Sep 10 18:35:32.240788 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:32.240768 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-d28xz_openshift-console-operator(3c15c29b-edff-4386-a50a-177b41560acd)\"" pod="openshift-console-operator/console-operator-85b454874b-d28xz" podUID="3c15c29b-edff-4386-a50a-177b41560acd" Sep 10 18:35:33.244967 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:33.244935 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d28xz_3c15c29b-edff-4386-a50a-177b41560acd/console-operator/2.log" Sep 10 18:35:38.761577 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:38.761503 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:38.761577 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:38.761557 2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:38.761989 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:38.761883 2564 scope.go:117] "RemoveContainer" containerID="8b13450c47aa192772f84e4ecb69b7d2261643dd795bcd78bb0881fedd52722e" Sep 10 18:35:38.762058 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:38.762039 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-d28xz_openshift-console-operator(3c15c29b-edff-4386-a50a-177b41560acd)\"" pod="openshift-console-operator/console-operator-85b454874b-d28xz" podUID="3c15c29b-edff-4386-a50a-177b41560acd" Sep 10 18:35:43.434287 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:43.434255 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gfdtb"] Sep 10 18:35:43.440689 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:43.440665 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gfdtb" Sep 10 18:35:43.442622 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:43.442589 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 10 18:35:43.442775 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:43.442620 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-69lqc\"" Sep 10 18:35:43.445869 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:43.445843 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gfdtb"] Sep 10 18:35:43.538349 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:43.538321 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0c39f34d-a437-4526-9abb-3cfe2d806bd4-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-gfdtb\" (UID: \"0c39f34d-a437-4526-9abb-3cfe2d806bd4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gfdtb" Sep 10 18:35:43.639381 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:43.639351 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0c39f34d-a437-4526-9abb-3cfe2d806bd4-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-gfdtb\" (UID: \"0c39f34d-a437-4526-9abb-3cfe2d806bd4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gfdtb" Sep 10 18:35:43.639547 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:43.639448 2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Sep 10 18:35:43.639547 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:43.639544 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c39f34d-a437-4526-9abb-3cfe2d806bd4-tls-certificates podName:0c39f34d-a437-4526-9abb-3cfe2d806bd4 nodeName:}" failed. No retries permitted until 2026-09-10 18:35:44.139528704 +0000 UTC m=+273.282831926 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/0c39f34d-a437-4526-9abb-3cfe2d806bd4-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-gfdtb" (UID: "0c39f34d-a437-4526-9abb-3cfe2d806bd4") : secret "prometheus-operator-admission-webhook-tls" not found Sep 10 18:35:44.143786 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:44.143751 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0c39f34d-a437-4526-9abb-3cfe2d806bd4-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-gfdtb\" (UID: \"0c39f34d-a437-4526-9abb-3cfe2d806bd4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gfdtb" Sep 10 18:35:44.146215 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:44.146185 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0c39f34d-a437-4526-9abb-3cfe2d806bd4-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-gfdtb\" (UID: \"0c39f34d-a437-4526-9abb-3cfe2d806bd4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gfdtb" Sep 10 18:35:44.350746 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:44.350698 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gfdtb" Sep 10 18:35:44.546581 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:44.546545 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:44.548990 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:44.548965 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb589d3-13f2-4138-948a-373b4e6b8dff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxt56\" (UID: \"9eb589d3-13f2-4138-948a-373b4e6b8dff\") " pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:44.616755 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:44.616723 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gfdtb"] Sep 10 18:35:44.619403 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:35:44.619367 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c39f34d_a437_4526_9abb_3cfe2d806bd4.slice/crio-7c701d11e38a5c770d4c2fad097088bdb88361be3f22b140a3e81781c19fb42b WatchSource:0}: Error finding container 7c701d11e38a5c770d4c2fad097088bdb88361be3f22b140a3e81781c19fb42b: Status 404 returned error can't find the container with id 7c701d11e38a5c770d4c2fad097088bdb88361be3f22b140a3e81781c19fb42b Sep 10 18:35:44.839332 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:44.839291 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jxt56" Sep 10 18:35:45.106123 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:45.106050 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jxt56"] Sep 10 18:35:45.109126 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:35:45.109099 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb589d3_13f2_4138_948a_373b4e6b8dff.slice/crio-6d908a85afafa55f56f7e4c750f2fa5ee908d672a3a4065d9e83f35f39042b40 WatchSource:0}: Error finding container 6d908a85afafa55f56f7e4c750f2fa5ee908d672a3a4065d9e83f35f39042b40: Status 404 returned error can't find the container with id 6d908a85afafa55f56f7e4c750f2fa5ee908d672a3a4065d9e83f35f39042b40 Sep 10 18:35:45.282082 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:45.282046 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jxt56" event={"ID":"9eb589d3-13f2-4138-948a-373b4e6b8dff","Type":"ContainerStarted","Data":"8075e4fdc9c83b6dec440c55ff5c665881a0c0ddf72f922a2def67c3b1f49b33"} Sep 10 18:35:45.282235 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:45.282137 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jxt56" event={"ID":"9eb589d3-13f2-4138-948a-373b4e6b8dff","Type":"ContainerStarted","Data":"6d908a85afafa55f56f7e4c750f2fa5ee908d672a3a4065d9e83f35f39042b40"} Sep 10 18:35:45.283104 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:45.283083 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gfdtb" event={"ID":"0c39f34d-a437-4526-9abb-3cfe2d806bd4","Type":"ContainerStarted","Data":"7c701d11e38a5c770d4c2fad097088bdb88361be3f22b140a3e81781c19fb42b"} Sep 10 18:35:47.291329 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.291293 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jxt56" event={"ID":"9eb589d3-13f2-4138-948a-373b4e6b8dff","Type":"ContainerStarted","Data":"5cc1b0a4a4e0f51763ca11cdf59952a2a86b4f9e2f70015149b2710e18e5ae46"} Sep 10 18:35:47.292724 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.292702 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gfdtb" event={"ID":"0c39f34d-a437-4526-9abb-3cfe2d806bd4","Type":"ContainerStarted","Data":"858a5f6f59a1768ea17baa45d2bb19b36d33f8c8bb852ef50260e10d380c734c"} Sep 10 18:35:47.292899 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.292879 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gfdtb" Sep 10 18:35:47.297351 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.297328 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gfdtb" Sep 10 18:35:47.305813 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.305764 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-gfdtb" podStartSLOduration=2.818651826 podStartE2EDuration="4.305751665s" totalImagesPullingTime="1.487099839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:35:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:35:47.305150342 +0000 UTC m=+276.448453587" watchObservedRunningTime="2026-09-10 18:35:47.305751665 +0000 UTC m=+276.449054944" Sep 10 18:35:47.496961 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.496931 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-94vpm"] Sep 10 18:35:47.499641 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.499612 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" Sep 10 18:35:47.501764 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.501736 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 10 18:35:47.501987 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.501970 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 10 18:35:47.502088 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.501974 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 18:35:47.502088 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.502063 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 18:35:47.502301 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.502281 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-t84sw\"" Sep 10 18:35:47.502414 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.502400 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 18:35:47.509530 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.509507 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-94vpm"] Sep 10 18:35:47.570457 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.570372 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4601784a-6817-45a3-b088-1c08b9410a01-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-94vpm\" (UID: \"4601784a-6817-45a3-b088-1c08b9410a01\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" Sep 10 18:35:47.570457 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.570421 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4601784a-6817-45a3-b088-1c08b9410a01-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-94vpm\" (UID: \"4601784a-6817-45a3-b088-1c08b9410a01\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" Sep 10 18:35:47.570705 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.570566 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pvbw\" (UniqueName: \"kubernetes.io/projected/4601784a-6817-45a3-b088-1c08b9410a01-kube-api-access-6pvbw\") pod \"prometheus-operator-6f4946db56-94vpm\" (UID: \"4601784a-6817-45a3-b088-1c08b9410a01\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" Sep 10 18:35:47.570705 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.570621 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4601784a-6817-45a3-b088-1c08b9410a01-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-94vpm\" (UID: \"4601784a-6817-45a3-b088-1c08b9410a01\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" Sep 10 18:35:47.671638 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.671599 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4601784a-6817-45a3-b088-1c08b9410a01-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-94vpm\" (UID: \"4601784a-6817-45a3-b088-1c08b9410a01\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" Sep 10 18:35:47.671810 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.671676 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4601784a-6817-45a3-b088-1c08b9410a01-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-94vpm\" (UID: \"4601784a-6817-45a3-b088-1c08b9410a01\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" Sep 10 18:35:47.671810 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.671704 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4601784a-6817-45a3-b088-1c08b9410a01-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-94vpm\" (UID: \"4601784a-6817-45a3-b088-1c08b9410a01\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" Sep 10 18:35:47.671810 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.671746 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pvbw\" (UniqueName: \"kubernetes.io/projected/4601784a-6817-45a3-b088-1c08b9410a01-kube-api-access-6pvbw\") pod \"prometheus-operator-6f4946db56-94vpm\" (UID: \"4601784a-6817-45a3-b088-1c08b9410a01\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" Sep 10 18:35:47.671810 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:47.671750 2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 10 18:35:47.672032 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:47.671825 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4601784a-6817-45a3-b088-1c08b9410a01-prometheus-operator-tls podName:4601784a-6817-45a3-b088-1c08b9410a01 nodeName:}" failed. No retries permitted until 2026-09-10 18:35:48.1718044 +0000 UTC m=+277.315107637 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/4601784a-6817-45a3-b088-1c08b9410a01-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-94vpm" (UID: "4601784a-6817-45a3-b088-1c08b9410a01") : secret "prometheus-operator-tls" not found Sep 10 18:35:47.672460 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.672438 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4601784a-6817-45a3-b088-1c08b9410a01-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-94vpm\" (UID: \"4601784a-6817-45a3-b088-1c08b9410a01\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" Sep 10 18:35:47.674581 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.674560 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4601784a-6817-45a3-b088-1c08b9410a01-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-94vpm\" (UID: \"4601784a-6817-45a3-b088-1c08b9410a01\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" Sep 10 18:35:47.682274 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:47.682254 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pvbw\" (UniqueName: \"kubernetes.io/projected/4601784a-6817-45a3-b088-1c08b9410a01-kube-api-access-6pvbw\") pod \"prometheus-operator-6f4946db56-94vpm\" (UID: \"4601784a-6817-45a3-b088-1c08b9410a01\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" Sep 10 18:35:48.176201 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:48.176164 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4601784a-6817-45a3-b088-1c08b9410a01-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-94vpm\" (UID: \"4601784a-6817-45a3-b088-1c08b9410a01\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" Sep 10 18:35:48.178392 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:48.178371 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4601784a-6817-45a3-b088-1c08b9410a01-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-94vpm\" (UID: \"4601784a-6817-45a3-b088-1c08b9410a01\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" Sep 10 18:35:48.297496 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:48.297446 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jxt56" event={"ID":"9eb589d3-13f2-4138-948a-373b4e6b8dff","Type":"ContainerStarted","Data":"a66caf480be30ce28c23c697dfad368d98cab333a194bd6c5fa087bf68cb4afa"} Sep 10 18:35:48.313080 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:48.313031 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jxt56" podStartSLOduration=33.435772616 podStartE2EDuration="36.313011725s" totalImagesPullingTime="2.877239109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:35:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:35:48.312221353 +0000 UTC m=+277.455524596" watchObservedRunningTime="2026-09-10 18:35:48.313011725 +0000 UTC m=+277.456314969" Sep 10 18:35:48.411279 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:48.411233 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" Sep 10 18:35:48.677351 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:48.677320 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-94vpm"] Sep 10 18:35:48.680147 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:35:48.680123 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4601784a_6817_45a3_b088_1c08b9410a01.slice/crio-bc714a1bcd7331134c3a75384ba2db11f3937432a97d563913fbe7fd29791e89 WatchSource:0}: Error finding container bc714a1bcd7331134c3a75384ba2db11f3937432a97d563913fbe7fd29791e89: Status 404 returned error can't find the container with id bc714a1bcd7331134c3a75384ba2db11f3937432a97d563913fbe7fd29791e89 Sep 10 18:35:49.301642 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:49.301605 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" event={"ID":"4601784a-6817-45a3-b088-1c08b9410a01","Type":"ContainerStarted","Data":"bc714a1bcd7331134c3a75384ba2db11f3937432a97d563913fbe7fd29791e89"} Sep 10 18:35:50.942833 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:50.942783 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" podUID="c6c62a12-04eb-40ce-b5cb-3a43fc562884" Sep 10 18:35:50.943191 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:50.942849 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" podUID="05dbe2c5-05cf-4b20-99e2-a50b46fe0252" Sep 10 18:35:50.943191 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:50.942849 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-nggdz" podUID="857e07ef-d71a-41f6-b863-b90126b0febb" Sep 10 18:35:51.311566 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:51.311529 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" event={"ID":"4601784a-6817-45a3-b088-1c08b9410a01","Type":"ContainerStarted","Data":"ebf113e3b4b7215f9222b23b01a449ef31f68b68e71184750e600d0a3ad5a5b9"} Sep 10 18:35:51.311566 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:51.311549 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nggdz" Sep 10 18:35:51.311771 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:51.311606 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" event={"ID":"4601784a-6817-45a3-b088-1c08b9410a01","Type":"ContainerStarted","Data":"7f74bbaf9d6ac19e6090271465a1f3834787ee44cac24861be2277a52c1af2d0"} Sep 10 18:35:51.311771 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:51.311754 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:35:51.311921 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:51.311836 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:35:51.326304 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:51.326262 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-94vpm" podStartSLOduration=2.731874521 podStartE2EDuration="4.326248917s" totalImagesPullingTime="1.594374396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:35:51.32474865 +0000 UTC m=+280.468051893" watchObservedRunningTime="2026-09-10 18:35:51.326248917 +0000 UTC m=+280.469552153" Sep 10 18:35:52.441569 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:52.441536 2564 scope.go:117] "RemoveContainer" containerID="8b13450c47aa192772f84e4ecb69b7d2261643dd795bcd78bb0881fedd52722e" Sep 10 18:35:52.886156 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:52.886123 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dp8n2"] Sep 10 18:35:52.889410 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:52.889394 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:52.891555 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:52.891532 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 18:35:52.891768 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:52.891751 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 18:35:52.891830 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:52.891783 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rjgtz\"" Sep 10 18:35:52.892342 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:52.892329 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 18:35:53.017544 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.017509 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32dba368-9cba-42ac-80bc-ee4ebe358f7a-sys\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.017680 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.017558 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-textfile\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.017680 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.017613 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-wtmp\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.017680 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.017635 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-tls\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.017680 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.017663 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32dba368-9cba-42ac-80bc-ee4ebe358f7a-metrics-client-ca\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.017834 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.017701 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.017834 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.017722 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgccb\" (UniqueName: \"kubernetes.io/projected/32dba368-9cba-42ac-80bc-ee4ebe358f7a-kube-api-access-fgccb\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.017834 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.017747 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32dba368-9cba-42ac-80bc-ee4ebe358f7a-root\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.017834 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.017775 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-accelerators-collector-config\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.118741 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.118705 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgccb\" (UniqueName: \"kubernetes.io/projected/32dba368-9cba-42ac-80bc-ee4ebe358f7a-kube-api-access-fgccb\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.118741 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.118745 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32dba368-9cba-42ac-80bc-ee4ebe358f7a-root\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.118976 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.118779 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-accelerators-collector-config\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.118976 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.118815 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32dba368-9cba-42ac-80bc-ee4ebe358f7a-sys\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.118976 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.118845 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-textfile\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.118976 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.118900 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32dba368-9cba-42ac-80bc-ee4ebe358f7a-root\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.118976 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.118907 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32dba368-9cba-42ac-80bc-ee4ebe358f7a-sys\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.119221 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.119033 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-wtmp\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.119221 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.119079 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-tls\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.119221 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.119106 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32dba368-9cba-42ac-80bc-ee4ebe358f7a-metrics-client-ca\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.119221 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.119146 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.119221 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.119168 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-textfile\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.119221 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.119195 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-wtmp\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.119426 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:53.119230 2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 10 18:35:53.119426 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:53.119290 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-tls podName:32dba368-9cba-42ac-80bc-ee4ebe358f7a nodeName:}" failed. No retries permitted until 2026-09-10 18:35:53.619274809 +0000 UTC m=+282.762578034 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-tls") pod "node-exporter-dp8n2" (UID: "32dba368-9cba-42ac-80bc-ee4ebe358f7a") : secret "node-exporter-tls" not found Sep 10 18:35:53.119579 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.119558 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-accelerators-collector-config\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.119613 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.119596 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32dba368-9cba-42ac-80bc-ee4ebe358f7a-metrics-client-ca\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.121410 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.121393 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.126346 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.126324 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgccb\" (UniqueName: \"kubernetes.io/projected/32dba368-9cba-42ac-80bc-ee4ebe358f7a-kube-api-access-fgccb\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.255278 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.255237 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-xknxg"] Sep 10 18:35:53.258526 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.258511 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-xknxg" Sep 10 18:35:53.260714 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.260689 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 18:35:53.260849 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.260689 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 18:35:53.260849 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.260697 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-c7kz4\"" Sep 10 18:35:53.272368 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.272334 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-xknxg"] Sep 10 18:35:53.319594 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.319567 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d28xz_3c15c29b-edff-4386-a50a-177b41560acd/console-operator/2.log" Sep 10 18:35:53.319720 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.319647 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-d28xz" event={"ID":"3c15c29b-edff-4386-a50a-177b41560acd","Type":"ContainerStarted","Data":"6f57ace279973fdbebbb4fc1d19134a35feec1a5fd10a5dfac8d9bd65a1a5972"} Sep 10 18:35:53.319937 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.319906 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:53.320317 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.320294 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2zvk\" (UniqueName: \"kubernetes.io/projected/9f9b30af-9874-4e85-9843-b0997e177371-kube-api-access-t2zvk\") pod \"downloads-85d98c998f-xknxg\" (UID: \"9f9b30af-9874-4e85-9843-b0997e177371\") " pod="openshift-console/downloads-85d98c998f-xknxg" Sep 10 18:35:53.324373 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.324353 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-d28xz" Sep 10 18:35:53.350010 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.349898 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-d28xz" podStartSLOduration=42.050543081 podStartE2EDuration="45.349881414s" totalImagesPullingTime="3.299338333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:35:53.349599408 +0000 UTC m=+282.492902669" watchObservedRunningTime="2026-09-10 18:35:53.349881414 +0000 UTC m=+282.493184672" Sep 10 18:35:53.421161 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.421123 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2zvk\" (UniqueName: \"kubernetes.io/projected/9f9b30af-9874-4e85-9843-b0997e177371-kube-api-access-t2zvk\") pod \"downloads-85d98c998f-xknxg\" (UID: \"9f9b30af-9874-4e85-9843-b0997e177371\") " pod="openshift-console/downloads-85d98c998f-xknxg" Sep 10 18:35:53.428038 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.428011 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2zvk\" (UniqueName: \"kubernetes.io/projected/9f9b30af-9874-4e85-9843-b0997e177371-kube-api-access-t2zvk\") pod \"downloads-85d98c998f-xknxg\" (UID: \"9f9b30af-9874-4e85-9843-b0997e177371\") " pod="openshift-console/downloads-85d98c998f-xknxg" Sep 10 18:35:53.568079 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.568000 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-xknxg" Sep 10 18:35:53.623028 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.623000 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-tls\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.625899 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.625872 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32dba368-9cba-42ac-80bc-ee4ebe358f7a-node-exporter-tls\") pod \"node-exporter-dp8n2\" (UID: \"32dba368-9cba-42ac-80bc-ee4ebe358f7a\") " pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.798282 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.798248 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dp8n2" Sep 10 18:35:53.810502 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:35:53.810455 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32dba368_9cba_42ac_80bc_ee4ebe358f7a.slice/crio-3e538f285a50bfe6fb3fa699dbe74a8febc117269ca33f84aa8f898c3d5963b8 WatchSource:0}: Error finding container 3e538f285a50bfe6fb3fa699dbe74a8febc117269ca33f84aa8f898c3d5963b8: Status 404 returned error can't find the container with id 3e538f285a50bfe6fb3fa699dbe74a8febc117269ca33f84aa8f898c3d5963b8 Sep 10 18:35:53.836177 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.836107 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-xknxg"] Sep 10 18:35:53.839455 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:35:53.839430 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f9b30af_9874_4e85_9843_b0997e177371.slice/crio-49407aac06aaff3cf81fa97f9a94fe704c59f851020099fe2c69a63e03f7e626 WatchSource:0}: Error finding container 49407aac06aaff3cf81fa97f9a94fe704c59f851020099fe2c69a63e03f7e626: Status 404 returned error can't find the container with id 49407aac06aaff3cf81fa97f9a94fe704c59f851020099fe2c69a63e03f7e626 Sep 10 18:35:53.902093 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.902060 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 18:35:53.906969 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.906948 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:53.910568 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.910507 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 18:35:53.910568 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.910536 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 18:35:53.910776 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.910604 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 18:35:53.910776 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.910628 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 18:35:53.910776 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.910643 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 18:35:53.910776 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.910703 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 18:35:53.910776 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.910705 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rdzqz\"" Sep 10 18:35:53.910776 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.910751 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 18:35:53.911061 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.910876 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 18:35:53.911283 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.911257 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 18:35:53.922713 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:53.922691 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 18:35:54.026598 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.026568 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.026598 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.026601 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.026843 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.026622 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.026843 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.026691 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8bb04f10-fe1b-4d07-bfed-890008ff5958-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.026843 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.026707 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.026843 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.026724 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb04f10-fe1b-4d07-bfed-890008ff5958-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.026843 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.026808 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msqch\" (UniqueName: \"kubernetes.io/projected/8bb04f10-fe1b-4d07-bfed-890008ff5958-kube-api-access-msqch\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.027084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.026848 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8bb04f10-fe1b-4d07-bfed-890008ff5958-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.027084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.026882 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bb04f10-fe1b-4d07-bfed-890008ff5958-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.027084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.026921 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-config-volume\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.027084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.026956 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-web-config\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.027084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.026995 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8bb04f10-fe1b-4d07-bfed-890008ff5958-config-out\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.027084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.027054 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.128195 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.128112 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.128195 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.128150 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.128195 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.128177 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.128450 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.128228 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8bb04f10-fe1b-4d07-bfed-890008ff5958-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.128450 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.128253 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.128450 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.128278 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb04f10-fe1b-4d07-bfed-890008ff5958-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.128450 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.128405 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msqch\" (UniqueName: \"kubernetes.io/projected/8bb04f10-fe1b-4d07-bfed-890008ff5958-kube-api-access-msqch\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.128450 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:35:54.128415 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8bb04f10-fe1b-4d07-bfed-890008ff5958-alertmanager-trusted-ca-bundle podName:8bb04f10-fe1b-4d07-bfed-890008ff5958 nodeName:}" failed. No retries permitted until 2026-09-10 18:35:54.628394918 +0000 UTC m=+283.771698162 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/8bb04f10-fe1b-4d07-bfed-890008ff5958-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "8bb04f10-fe1b-4d07-bfed-890008ff5958") : configmap references non-existent config key: ca-bundle.crt Sep 10 18:35:54.128753 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.128454 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8bb04f10-fe1b-4d07-bfed-890008ff5958-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.128753 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.128513 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bb04f10-fe1b-4d07-bfed-890008ff5958-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.128753 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.128553 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-config-volume\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.128753 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.128596 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-web-config\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.128753 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.128644 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8bb04f10-fe1b-4d07-bfed-890008ff5958-config-out\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.128753 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.128691 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.129291 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.129269 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8bb04f10-fe1b-4d07-bfed-890008ff5958-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.130062 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.130035 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bb04f10-fe1b-4d07-bfed-890008ff5958-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.131381 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.131356 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8bb04f10-fe1b-4d07-bfed-890008ff5958-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.131465 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.131369 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.131602 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.131581 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-web-config\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.131984 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.131959 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.132340 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.132321 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.132435 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.132417 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8bb04f10-fe1b-4d07-bfed-890008ff5958-config-out\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.132936 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.132917 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.133025 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.132983 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.133025 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.132984 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-config-volume\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.135876 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.135855 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msqch\" (UniqueName: \"kubernetes.io/projected/8bb04f10-fe1b-4d07-bfed-890008ff5958-kube-api-access-msqch\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.325392 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.325350 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-xknxg" event={"ID":"9f9b30af-9874-4e85-9843-b0997e177371","Type":"ContainerStarted","Data":"49407aac06aaff3cf81fa97f9a94fe704c59f851020099fe2c69a63e03f7e626"} Sep 10 18:35:54.326780 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.326746 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dp8n2" event={"ID":"32dba368-9cba-42ac-80bc-ee4ebe358f7a","Type":"ContainerStarted","Data":"3e538f285a50bfe6fb3fa699dbe74a8febc117269ca33f84aa8f898c3d5963b8"} Sep 10 18:35:54.633266 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.633229 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb04f10-fe1b-4d07-bfed-890008ff5958-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.634327 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.634295 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb04f10-fe1b-4d07-bfed-890008ff5958-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.734193 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.734158 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5tq6l\" (UID: \"c6c62a12-04eb-40ce-b5cb-3a43fc562884\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:35:54.737265 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.737237 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c6c62a12-04eb-40ce-b5cb-3a43fc562884-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-5tq6l\" (UID: \"c6c62a12-04eb-40ce-b5cb-3a43fc562884\") " pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:35:54.817136 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.817102 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:35:54.835171 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.835141 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:35:54.835376 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.835204 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:35:54.838191 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.838017 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/857e07ef-d71a-41f6-b863-b90126b0febb-metrics-tls\") pod \"dns-default-nggdz\" (UID: \"857e07ef-d71a-41f6-b863-b90126b0febb\") " pod="openshift-dns/dns-default-nggdz" Sep 10 18:35:54.838191 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.838024 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls\") pod \"image-registry-84ccb96897-cld7r\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:35:54.914945 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.914860 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4lj58\"" Sep 10 18:35:54.914945 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.914862 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-rmh66\"" Sep 10 18:35:54.915127 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.914869 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xgqmb\"" Sep 10 18:35:54.922724 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.922529 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nggdz" Sep 10 18:35:54.922724 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.922639 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:35:54.922724 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:54.922702 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" Sep 10 18:35:55.129871 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:55.129831 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 18:35:55.133443 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:35:55.133420 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bb04f10_fe1b_4d07_bfed_890008ff5958.slice/crio-9e5cb75236928925568799cb6879e89273be7ce4021e14ca262854c08d678376 WatchSource:0}: Error finding container 9e5cb75236928925568799cb6879e89273be7ce4021e14ca262854c08d678376: Status 404 returned error can't find the container with id 9e5cb75236928925568799cb6879e89273be7ce4021e14ca262854c08d678376 Sep 10 18:35:55.210470 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:55.210447 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nggdz"] Sep 10 18:35:55.213162 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:35:55.213139 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod857e07ef_d71a_41f6_b863_b90126b0febb.slice/crio-2da4b088c8bc957d6d34b68f15df1e10594725fa59f49de3362153db82f75a1d WatchSource:0}: Error finding container 2da4b088c8bc957d6d34b68f15df1e10594725fa59f49de3362153db82f75a1d: Status 404 returned error can't find the container with id 2da4b088c8bc957d6d34b68f15df1e10594725fa59f49de3362153db82f75a1d Sep 10 18:35:55.221952 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:55.221833 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84ccb96897-cld7r"] Sep 10 18:35:55.224628 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:35:55.224602 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05dbe2c5_05cf_4b20_99e2_a50b46fe0252.slice/crio-ac02dacfc00d25d0081a9f5d06d2532b56a1d5f35ba01683e8a91c35468a4b78 WatchSource:0}: Error finding container ac02dacfc00d25d0081a9f5d06d2532b56a1d5f35ba01683e8a91c35468a4b78: Status 404 returned error can't find the container with id ac02dacfc00d25d0081a9f5d06d2532b56a1d5f35ba01683e8a91c35468a4b78 Sep 10 18:35:55.244874 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:55.244851 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-5tq6l"] Sep 10 18:35:55.248180 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:35:55.248151 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6c62a12_04eb_40ce_b5cb_3a43fc562884.slice/crio-aaf80bc8c26446a0a4d92a6e29e0ede121244bfd6bcb396e930dd6ae6d2a4253 WatchSource:0}: Error finding container aaf80bc8c26446a0a4d92a6e29e0ede121244bfd6bcb396e930dd6ae6d2a4253: Status 404 returned error can't find the container with id aaf80bc8c26446a0a4d92a6e29e0ede121244bfd6bcb396e930dd6ae6d2a4253 Sep 10 18:35:55.337563 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:55.337520 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" event={"ID":"05dbe2c5-05cf-4b20-99e2-a50b46fe0252","Type":"ContainerStarted","Data":"1ef6da4b54ae5eaf7aede54e0b449068241967f9cba15883549863ddab3fea04"} Sep 10 18:35:55.337731 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:55.337615 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" event={"ID":"05dbe2c5-05cf-4b20-99e2-a50b46fe0252","Type":"ContainerStarted","Data":"ac02dacfc00d25d0081a9f5d06d2532b56a1d5f35ba01683e8a91c35468a4b78"} Sep 10 18:35:55.337731 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:55.337643 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:35:55.339927 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:55.339899 2564 generic.go:356] "Generic (PLEG): container finished" podID="32dba368-9cba-42ac-80bc-ee4ebe358f7a" containerID="a6ea6cc97ed336affdeea206baf652d9d1ca7c2982aabfd8e836c960e5f680c0" exitCode=0 Sep 10 18:35:55.340155 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:55.339994 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dp8n2" event={"ID":"32dba368-9cba-42ac-80bc-ee4ebe358f7a","Type":"ContainerDied","Data":"a6ea6cc97ed336affdeea206baf652d9d1ca7c2982aabfd8e836c960e5f680c0"} Sep 10 18:35:55.341864 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:55.341843 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" event={"ID":"c6c62a12-04eb-40ce-b5cb-3a43fc562884","Type":"ContainerStarted","Data":"aaf80bc8c26446a0a4d92a6e29e0ede121244bfd6bcb396e930dd6ae6d2a4253"} Sep 10 18:35:55.343605 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:55.343579 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nggdz" event={"ID":"857e07ef-d71a-41f6-b863-b90126b0febb","Type":"ContainerStarted","Data":"2da4b088c8bc957d6d34b68f15df1e10594725fa59f49de3362153db82f75a1d"} Sep 10 18:35:55.345341 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:55.345319 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8bb04f10-fe1b-4d07-bfed-890008ff5958","Type":"ContainerStarted","Data":"9e5cb75236928925568799cb6879e89273be7ce4021e14ca262854c08d678376"} Sep 10 18:35:55.355363 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:55.355197 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" podStartSLOduration=284.355179239 podStartE2EDuration="4m44.355179239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:31:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:35:55.354022551 +0000 UTC m=+284.497325808" watchObservedRunningTime="2026-09-10 18:35:55.355179239 +0000 UTC m=+284.498482484" Sep 10 18:35:56.353635 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:56.353528 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dp8n2" event={"ID":"32dba368-9cba-42ac-80bc-ee4ebe358f7a","Type":"ContainerStarted","Data":"9acfe66d1850caaa90cbe59ac28333adcc73046c0aade30a5380d347fb34cb13"} Sep 10 18:35:56.353635 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:56.353590 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dp8n2" event={"ID":"32dba368-9cba-42ac-80bc-ee4ebe358f7a","Type":"ContainerStarted","Data":"68c3e394c2cc00a8079a539ae6d3b815812556719310a2256a4af351f8ad08ab"} Sep 10 18:35:56.371380 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:56.370899 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dp8n2" podStartSLOduration=3.25067979 podStartE2EDuration="4.370879918s" totalImagesPullingTime="1.027644514s" totalInitContainerRuntime="92.555614ms" isStatefulPod=true podCreationTimestamp="2026-09-10 18:35:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:35:56.369736761 +0000 UTC m=+285.513040007" watchObservedRunningTime="2026-09-10 18:35:56.370879918 +0000 UTC m=+285.514183164" Sep 10 18:35:57.623478 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:57.623446 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-26vff"] Sep 10 18:35:57.626627 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:57.626607 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-26vff" Sep 10 18:35:57.628476 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:57.628444 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 10 18:35:57.628644 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:57.628623 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-n8k6w\"" Sep 10 18:35:57.636324 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:57.636302 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-26vff"] Sep 10 18:35:57.762808 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:57.762774 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdhrz\" (UniqueName: \"kubernetes.io/projected/c3b212e2-aa57-4cae-a509-d4cd8a8e133a-kube-api-access-gdhrz\") pod \"monitoring-plugin-85bbbbbf4-26vff\" (UID: \"c3b212e2-aa57-4cae-a509-d4cd8a8e133a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-26vff" Sep 10 18:35:57.762808 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:57.762817 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c3b212e2-aa57-4cae-a509-d4cd8a8e133a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-26vff\" (UID: \"c3b212e2-aa57-4cae-a509-d4cd8a8e133a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-26vff" Sep 10 18:35:57.864037 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:57.863952 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdhrz\" (UniqueName: \"kubernetes.io/projected/c3b212e2-aa57-4cae-a509-d4cd8a8e133a-kube-api-access-gdhrz\") pod \"monitoring-plugin-85bbbbbf4-26vff\" (UID: \"c3b212e2-aa57-4cae-a509-d4cd8a8e133a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-26vff" Sep 10 18:35:57.864037 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:57.863998 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c3b212e2-aa57-4cae-a509-d4cd8a8e133a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-26vff\" (UID: \"c3b212e2-aa57-4cae-a509-d4cd8a8e133a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-26vff" Sep 10 18:35:57.866715 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:57.866691 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c3b212e2-aa57-4cae-a509-d4cd8a8e133a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-26vff\" (UID: \"c3b212e2-aa57-4cae-a509-d4cd8a8e133a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-26vff" Sep 10 18:35:57.871025 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:57.870997 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdhrz\" (UniqueName: \"kubernetes.io/projected/c3b212e2-aa57-4cae-a509-d4cd8a8e133a-kube-api-access-gdhrz\") pod \"monitoring-plugin-85bbbbbf4-26vff\" (UID: \"c3b212e2-aa57-4cae-a509-d4cd8a8e133a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-26vff" Sep 10 18:35:57.935841 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:57.935803 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-26vff" Sep 10 18:35:58.210153 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:58.210077 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-26vff"] Sep 10 18:35:58.213152 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:35:58.213124 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3b212e2_aa57_4cae_a509_d4cd8a8e133a.slice/crio-41a8fece4ea3b11fa24870100ea833dfbeace5f3dfb1e21781e023b81e7ee8b0 WatchSource:0}: Error finding container 41a8fece4ea3b11fa24870100ea833dfbeace5f3dfb1e21781e023b81e7ee8b0: Status 404 returned error can't find the container with id 41a8fece4ea3b11fa24870100ea833dfbeace5f3dfb1e21781e023b81e7ee8b0 Sep 10 18:35:58.363109 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:58.363071 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" event={"ID":"c6c62a12-04eb-40ce-b5cb-3a43fc562884","Type":"ContainerStarted","Data":"330bb0532a6820a1071dcec63be72f20e90e4f55a16720db8b5c3468f3c3708c"} Sep 10 18:35:58.365285 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:58.365260 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nggdz" event={"ID":"857e07ef-d71a-41f6-b863-b90126b0febb","Type":"ContainerStarted","Data":"39520d20c207b6d4714fbb75e1637f3976335cac8c36ff6163a70c2c22e9eca2"} Sep 10 18:35:58.365412 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:58.365311 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nggdz" event={"ID":"857e07ef-d71a-41f6-b863-b90126b0febb","Type":"ContainerStarted","Data":"2b314437923a6c481d70c4057cc98b7c0d6cfda417746760ce7e7269b432c90f"} Sep 10 18:35:58.365412 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:58.365398 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nggdz" Sep 10 18:35:58.367266 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:58.367238 2564 generic.go:356] "Generic (PLEG): container finished" podID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerID="0d3585d1a63cf7fdf41b4160a9cb52b9c0b210d0b0f9cfc2994b8eb8d3f1777c" exitCode=0 Sep 10 18:35:58.367378 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:58.367322 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8bb04f10-fe1b-4d07-bfed-890008ff5958","Type":"ContainerDied","Data":"0d3585d1a63cf7fdf41b4160a9cb52b9c0b210d0b0f9cfc2994b8eb8d3f1777c"} Sep 10 18:35:58.373646 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:58.373619 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-26vff" event={"ID":"c3b212e2-aa57-4cae-a509-d4cd8a8e133a","Type":"ContainerStarted","Data":"41a8fece4ea3b11fa24870100ea833dfbeace5f3dfb1e21781e023b81e7ee8b0"} Sep 10 18:35:58.377942 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:58.377906 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-5tq6l" podStartSLOduration=282.339861755 podStartE2EDuration="4m44.377890266s" totalImagesPullingTime="2.038028511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:31:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:35:58.37687244 +0000 UTC m=+287.520175685" watchObservedRunningTime="2026-09-10 18:35:58.377890266 +0000 UTC m=+287.521193512" Sep 10 18:35:58.391399 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:58.391358 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nggdz" podStartSLOduration=252.268788085 podStartE2EDuration="4m14.391341506s" totalImagesPullingTime="2.122553421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:31:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:35:58.390684081 +0000 UTC m=+287.533987325" watchObservedRunningTime="2026-09-10 18:35:58.391341506 +0000 UTC m=+287.534644750" Sep 10 18:35:59.063428 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.062358 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 18:35:59.069846 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.069820 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.073825 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.072450 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 18:35:59.073825 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.072556 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 18:35:59.073825 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.072450 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 18:35:59.074328 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.074311 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 18:35:59.075240 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.074388 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 18:35:59.075240 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.074471 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 18:35:59.076085 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.074549 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bnfnenbtu9m44\"" Sep 10 18:35:59.076085 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.074587 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 18:35:59.076085 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.075751 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 18:35:59.076085 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.075781 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 18:35:59.076085 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.074590 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 18:35:59.076085 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.074754 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fldfc\"" Sep 10 18:35:59.076085 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.074789 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 18:35:59.076085 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.074816 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 18:35:59.076085 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.074716 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 18:35:59.076601 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.076161 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 18:35:59.078507 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.077582 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 18:35:59.079589 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.079435 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 18:35:59.176199 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176163 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176199 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176212 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176439 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176242 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176439 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176298 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176439 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176348 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176439 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176371 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b29t\" (UniqueName: \"kubernetes.io/projected/a4541aa2-c3d6-4af7-bd34-de202b1375b2-kube-api-access-6b29t\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176439 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176400 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176640 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176459 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-config\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176640 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176534 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-web-config\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176640 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176569 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176640 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176598 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176792 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176642 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176792 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176667 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176792 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176696 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176792 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176771 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4541aa2-c3d6-4af7-bd34-de202b1375b2-config-out\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176975 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176810 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176975 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176850 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176975 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176930 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.176975 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176962 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.177131 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.176997 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4541aa2-c3d6-4af7-bd34-de202b1375b2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.277878 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.277841 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-config\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.278083 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.277907 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-web-config\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.278083 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.277937 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.278083 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.277961 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.278258 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.278131 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.278258 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.278172 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.278258 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.278211 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.278258 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.278252 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4541aa2-c3d6-4af7-bd34-de202b1375b2-config-out\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.278447 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.278277 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.278447 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.278314 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.278447 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.278372 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.278447 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.278402 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.280109 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.280037 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.280640 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.280606 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4541aa2-c3d6-4af7-bd34-de202b1375b2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.280755 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.280658 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.280755 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.280693 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.280755 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.280723 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.280932 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.280758 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.280932 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.280767 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.280932 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.280807 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.280932 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.280833 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b29t\" (UniqueName: \"kubernetes.io/projected/a4541aa2-c3d6-4af7-bd34-de202b1375b2-kube-api-access-6b29t\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.280932 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.280884 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.281669 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.281641 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.281669 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.281862 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.281669 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.282048 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.281669 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.282094 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.281669 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.282300 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.281669 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.282652 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.284789 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.284549 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4541aa2-c3d6-4af7-bd34-de202b1375b2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.285058 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.285032 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.285155 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.285098 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.286037 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.286013 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.286473 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.286321 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.286473 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.286362 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-config\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.286473 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.286429 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4541aa2-c3d6-4af7-bd34-de202b1375b2-config-out\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.287895 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.287433 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.287895 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.287707 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.287895 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.287857 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-web-config\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.288109 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.287980 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.289601 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.289578 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b29t\" (UniqueName: \"kubernetes.io/projected/a4541aa2-c3d6-4af7-bd34-de202b1375b2-kube-api-access-6b29t\") pod \"prometheus-k8s-0\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:35:59.387366 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:35:59.387290 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:36:00.079874 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:00.079847 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 18:36:00.082027 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:36:00.081994 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4541aa2_c3d6_4af7_bd34_de202b1375b2.slice/crio-8094bcad3cd8f33296eb4b5c8fdb23fa895a5778ed323c261820c68bcae83f26 WatchSource:0}: Error finding container 8094bcad3cd8f33296eb4b5c8fdb23fa895a5778ed323c261820c68bcae83f26: Status 404 returned error can't find the container with id 8094bcad3cd8f33296eb4b5c8fdb23fa895a5778ed323c261820c68bcae83f26 Sep 10 18:36:00.213644 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:36:00.213618 2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4541aa2_c3d6_4af7_bd34_de202b1375b2.slice/crio-c4cf59b4392bf21b7eec9b4b84b68cf594940aeeef1e8e3f427ceaae324ee0e0.scope\": RecentStats: unable to find data in memory cache]" Sep 10 18:36:00.390714 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:00.390627 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-26vff" event={"ID":"c3b212e2-aa57-4cae-a509-d4cd8a8e133a","Type":"ContainerStarted","Data":"fb9bbd1651884182a566d5cae49df130b50b506f7e6d32fef36429af558e7b67"} Sep 10 18:36:00.390863 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:00.390721 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-26vff" Sep 10 18:36:00.392880 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:00.392829 2564 generic.go:356] "Generic (PLEG): container finished" podID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerID="c4cf59b4392bf21b7eec9b4b84b68cf594940aeeef1e8e3f427ceaae324ee0e0" exitCode=0 Sep 10 18:36:00.392880 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:00.392866 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4541aa2-c3d6-4af7-bd34-de202b1375b2","Type":"ContainerDied","Data":"c4cf59b4392bf21b7eec9b4b84b68cf594940aeeef1e8e3f427ceaae324ee0e0"} Sep 10 18:36:00.393046 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:00.392924 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4541aa2-c3d6-4af7-bd34-de202b1375b2","Type":"ContainerStarted","Data":"8094bcad3cd8f33296eb4b5c8fdb23fa895a5778ed323c261820c68bcae83f26"} Sep 10 18:36:00.397665 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:00.397639 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-26vff" Sep 10 18:36:00.404940 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:00.404818 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-26vff" podStartSLOduration=1.771176673 podStartE2EDuration="3.404799404s" totalImagesPullingTime="1.633622731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:35:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:36:00.403775418 +0000 UTC m=+289.547078687" watchObservedRunningTime="2026-09-10 18:36:00.404799404 +0000 UTC m=+289.548102649" Sep 10 18:36:01.402542 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:01.402448 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8bb04f10-fe1b-4d07-bfed-890008ff5958","Type":"ContainerStarted","Data":"9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490"} Sep 10 18:36:01.402932 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:01.402550 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8bb04f10-fe1b-4d07-bfed-890008ff5958","Type":"ContainerStarted","Data":"55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818"} Sep 10 18:36:01.402932 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:01.402599 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8bb04f10-fe1b-4d07-bfed-890008ff5958","Type":"ContainerStarted","Data":"73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb"} Sep 10 18:36:01.402932 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:01.402641 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8bb04f10-fe1b-4d07-bfed-890008ff5958","Type":"ContainerStarted","Data":"2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7"} Sep 10 18:36:01.402932 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:01.402686 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8bb04f10-fe1b-4d07-bfed-890008ff5958","Type":"ContainerStarted","Data":"60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945"} Sep 10 18:36:02.412378 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:02.412342 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8bb04f10-fe1b-4d07-bfed-890008ff5958","Type":"ContainerStarted","Data":"773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e"} Sep 10 18:36:02.456273 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:02.456223 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.142937532 podStartE2EDuration="9.456203378s" totalImagesPullingTime="5.287690478s" totalInitContainerRuntime="25.575368ms" isStatefulPod=false podCreationTimestamp="2026-09-10 18:35:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:36:02.45480307 +0000 UTC m=+291.598106314" watchObservedRunningTime="2026-09-10 18:36:02.456203378 +0000 UTC m=+291.599506624" Sep 10 18:36:04.443628 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:36:04.443577 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6dxjk" podUID="99da651c-7b27-49ea-b49a-7f72caf21100" Sep 10 18:36:05.430716 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:05.430673 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4541aa2-c3d6-4af7-bd34-de202b1375b2","Type":"ContainerStarted","Data":"3488f8c1468076f34ab5559e3a428f052f3ef62269d89808d108e9021749227c"} Sep 10 18:36:05.430882 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:05.430765 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4541aa2-c3d6-4af7-bd34-de202b1375b2","Type":"ContainerStarted","Data":"1c34018c2e065dc17b103cef43c7c18995e35e7891ad5a91f0748768691972d5"} Sep 10 18:36:08.381497 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:08.381445 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nggdz" Sep 10 18:36:13.980150 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:13.980133 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d28xz_3c15c29b-edff-4386-a50a-177b41560acd/console-operator/2.log" Sep 10 18:36:13.980514 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:13.980313 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d28xz_3c15c29b-edff-4386-a50a-177b41560acd/console-operator/2.log" Sep 10 18:36:13.989669 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:13.989650 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/ovn-acl-logging/0.log" Sep 10 18:36:13.990196 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:13.990176 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/ovn-acl-logging/0.log" Sep 10 18:36:13.994916 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:13.994851 2564 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 18:36:14.475169 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:14.475125 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-xknxg" event={"ID":"9f9b30af-9874-4e85-9843-b0997e177371","Type":"ContainerStarted","Data":"3c0ee9af5176fdf43354a57f7f9bbc9bca6f1c44ed03b331e8dbd42126a6872e"} Sep 10 18:36:14.475356 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:14.475313 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-xknxg" Sep 10 18:36:14.480140 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:14.480060 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4541aa2-c3d6-4af7-bd34-de202b1375b2","Type":"ContainerStarted","Data":"6eac00504c049c61218406ddec8af210b6b6b0886ec2349e13c920710c27281b"} Sep 10 18:36:14.480140 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:14.480136 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4541aa2-c3d6-4af7-bd34-de202b1375b2","Type":"ContainerStarted","Data":"22dbe8b6b490a1e8f57c31e7c3a6e4e8bd505e6651fc9ab62513e616e102ea48"} Sep 10 18:36:14.480322 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:14.480173 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4541aa2-c3d6-4af7-bd34-de202b1375b2","Type":"ContainerStarted","Data":"9aebe97131ae59cb34866a74300421d058c69e1426400234a2f7242e3961078b"} Sep 10 18:36:14.480322 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:14.480202 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4541aa2-c3d6-4af7-bd34-de202b1375b2","Type":"ContainerStarted","Data":"877ab1e4af5d644a113a380d16d3cc07804399eba8089562f9032b13431c210b"} Sep 10 18:36:14.482602 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:14.482577 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-xknxg" Sep 10 18:36:14.490848 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:14.490793 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-xknxg" podStartSLOduration=1.321698867 podStartE2EDuration="21.490777138s" totalImagesPullingTime="20.169078271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:35:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:36:14.4888896 +0000 UTC m=+303.632192845" watchObservedRunningTime="2026-09-10 18:36:14.490777138 +0000 UTC m=+303.634080382" Sep 10 18:36:14.513755 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:14.513697 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.060815012 podStartE2EDuration="15.513675655s" totalImagesPullingTime="13.428565168s" totalInitContainerRuntime="24.295475ms" isStatefulPod=false podCreationTimestamp="2026-09-10 18:35:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:36:14.51159024 +0000 UTC m=+303.654893484" watchObservedRunningTime="2026-09-10 18:36:14.513675655 +0000 UTC m=+303.656978904" Sep 10 18:36:14.928409 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:14.928327 2564 patch_prober.go:28] interesting pod/image-registry-84ccb96897-cld7r container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 10 18:36:14.928409 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:14.928383 2564 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" podUID="05dbe2c5-05cf-4b20-99e2-a50b46fe0252" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 18:36:16.359026 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:16.358992 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:36:16.441731 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:16.441696 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:36:19.388636 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:19.388593 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:36:25.446731 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:25.446693 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84ccb96897-cld7r"] Sep 10 18:36:28.543972 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:28.543940 2564 generic.go:356] "Generic (PLEG): container finished" podID="418a0ba4-c771-43aa-bc0c-baf2b0b4f34f" containerID="61ce64b9e82adc1a8c372dbdf0dea0f1668be94f96718fee50e3667673417b93" exitCode=0 Sep 10 18:36:28.544449 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:28.544004 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9" event={"ID":"418a0ba4-c771-43aa-bc0c-baf2b0b4f34f","Type":"ContainerDied","Data":"61ce64b9e82adc1a8c372dbdf0dea0f1668be94f96718fee50e3667673417b93"} Sep 10 18:36:28.544449 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:28.544437 2564 scope.go:117] "RemoveContainer" containerID="61ce64b9e82adc1a8c372dbdf0dea0f1668be94f96718fee50e3667673417b93" Sep 10 18:36:29.549533 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:29.549498 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-sm6d9" event={"ID":"418a0ba4-c771-43aa-bc0c-baf2b0b4f34f","Type":"ContainerStarted","Data":"ccc820c4f4d1342da05c6a93ddc852ead2eb0f293ec7a92f79b3f285ab8a4225"} Sep 10 18:36:43.606087 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:43.606055 2564 generic.go:356] "Generic (PLEG): container finished" podID="89753db4-e14c-4c10-9b8a-773a684985e3" containerID="985e50f8506ae63bddf960194a10f4d81417ecacf3ac0ebc4cf5d3460a879966" exitCode=0 Sep 10 18:36:43.606476 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:43.606107 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx" event={"ID":"89753db4-e14c-4c10-9b8a-773a684985e3","Type":"ContainerDied","Data":"985e50f8506ae63bddf960194a10f4d81417ecacf3ac0ebc4cf5d3460a879966"} Sep 10 18:36:43.606476 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:43.606406 2564 scope.go:117] "RemoveContainer" containerID="985e50f8506ae63bddf960194a10f4d81417ecacf3ac0ebc4cf5d3460a879966" Sep 10 18:36:43.950593 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:43.950519 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-84ccb96897-cld7r_05dbe2c5-05cf-4b20-99e2-a50b46fe0252/registry/0.log" Sep 10 18:36:44.137762 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:44.137739 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cd2ls_96332d76-e827-4612-a6b5-bd9d31fe347e/node-ca/0.log" Sep 10 18:36:44.611646 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:44.611617 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-rzdsx" event={"ID":"89753db4-e14c-4c10-9b8a-773a684985e3","Type":"ContainerStarted","Data":"9970a7a67ea30ba54c08f849f0e8fa179119af0ac8ee6f4a8ae3ed493f48f169"} Sep 10 18:36:45.338692 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:45.338660 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zkgwb_cc9720ca-244c-4b07-88e5-e95fbeae172d/serve-healthcheck-canary/0.log" Sep 10 18:36:50.466593 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.466533 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" podUID="05dbe2c5-05cf-4b20-99e2-a50b46fe0252" containerName="registry" containerID="cri-o://1ef6da4b54ae5eaf7aede54e0b449068241967f9cba15883549863ddab3fea04" gracePeriod=30 Sep 10 18:36:50.637780 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.637748 2564 generic.go:356] "Generic (PLEG): container finished" podID="05dbe2c5-05cf-4b20-99e2-a50b46fe0252" containerID="1ef6da4b54ae5eaf7aede54e0b449068241967f9cba15883549863ddab3fea04" exitCode=0 Sep 10 18:36:50.637921 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.637795 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" event={"ID":"05dbe2c5-05cf-4b20-99e2-a50b46fe0252","Type":"ContainerDied","Data":"1ef6da4b54ae5eaf7aede54e0b449068241967f9cba15883549863ddab3fea04"} Sep 10 18:36:50.748234 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.748173 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:36:50.864621 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.864588 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-image-registry-private-configuration\") pod \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " Sep 10 18:36:50.864621 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.864625 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-trusted-ca\") pod \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " Sep 10 18:36:50.864906 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.864651 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls\") pod \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " Sep 10 18:36:50.864906 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.864673 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-ca-trust-extracted\") pod \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " Sep 10 18:36:50.864906 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.864807 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-certificates\") pod \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " Sep 10 18:36:50.864906 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.864849 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-bound-sa-token\") pod \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " Sep 10 18:36:50.865079 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.864926 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-installation-pull-secrets\") pod \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " Sep 10 18:36:50.865079 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.864976 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l4s9\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-kube-api-access-8l4s9\") pod \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\" (UID: \"05dbe2c5-05cf-4b20-99e2-a50b46fe0252\") " Sep 10 18:36:50.865248 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.865208 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "05dbe2c5-05cf-4b20-99e2-a50b46fe0252" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:36:50.865379 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.865266 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "05dbe2c5-05cf-4b20-99e2-a50b46fe0252" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:36:50.867115 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.867089 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "05dbe2c5-05cf-4b20-99e2-a50b46fe0252" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:36:50.867537 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.867477 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "05dbe2c5-05cf-4b20-99e2-a50b46fe0252" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:36:50.867697 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.867675 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-kube-api-access-8l4s9" (OuterVolumeSpecName: "kube-api-access-8l4s9") pod "05dbe2c5-05cf-4b20-99e2-a50b46fe0252" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252"). InnerVolumeSpecName "kube-api-access-8l4s9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:36:50.867756 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.867690 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "05dbe2c5-05cf-4b20-99e2-a50b46fe0252" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:36:50.867756 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.867713 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "05dbe2c5-05cf-4b20-99e2-a50b46fe0252" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:36:50.873951 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.873929 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "05dbe2c5-05cf-4b20-99e2-a50b46fe0252" (UID: "05dbe2c5-05cf-4b20-99e2-a50b46fe0252"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:36:50.966079 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.966035 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-tls\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:36:50.966079 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.966074 2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-ca-trust-extracted\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:36:50.966079 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.966086 2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-registry-certificates\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:36:50.966079 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.966095 2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-bound-sa-token\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:36:50.966323 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.966105 2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-installation-pull-secrets\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:36:50.966323 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.966114 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l4s9\" (UniqueName: \"kubernetes.io/projected/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-kube-api-access-8l4s9\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:36:50.966323 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.966142 2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-image-registry-private-configuration\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:36:50.966323 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:50.966152 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05dbe2c5-05cf-4b20-99e2-a50b46fe0252-trusted-ca\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:36:51.642442 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:51.642407 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" event={"ID":"05dbe2c5-05cf-4b20-99e2-a50b46fe0252","Type":"ContainerDied","Data":"ac02dacfc00d25d0081a9f5d06d2532b56a1d5f35ba01683e8a91c35468a4b78"} Sep 10 18:36:51.642442 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:51.642433 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84ccb96897-cld7r" Sep 10 18:36:51.642959 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:51.642506 2564 scope.go:117] "RemoveContainer" containerID="1ef6da4b54ae5eaf7aede54e0b449068241967f9cba15883549863ddab3fea04" Sep 10 18:36:51.657024 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:51.657000 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84ccb96897-cld7r"] Sep 10 18:36:51.660537 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:51.660516 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-84ccb96897-cld7r"] Sep 10 18:36:53.446419 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:53.446386 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05dbe2c5-05cf-4b20-99e2-a50b46fe0252" path="/var/lib/kubelet/pods/05dbe2c5-05cf-4b20-99e2-a50b46fe0252/volumes" Sep 10 18:36:59.388393 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:59.388357 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:36:59.407920 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:59.407883 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:36:59.694465 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:36:59.694386 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:13.146649 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:13.146597 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 18:37:13.147087 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:13.146983 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="alertmanager" containerID="cri-o://60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945" gracePeriod=120 Sep 10 18:37:13.147087 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:13.147068 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="kube-rbac-proxy-metric" containerID="cri-o://9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490" gracePeriod=120 Sep 10 18:37:13.147200 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:13.147067 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="kube-rbac-proxy-web" containerID="cri-o://73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb" gracePeriod=120 Sep 10 18:37:13.147200 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:13.147096 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="config-reloader" containerID="cri-o://2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7" gracePeriod=120 Sep 10 18:37:13.147200 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:13.147128 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="prom-label-proxy" containerID="cri-o://773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e" gracePeriod=120 Sep 10 18:37:13.147200 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:13.147071 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="kube-rbac-proxy" containerID="cri-o://55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818" gracePeriod=120 Sep 10 18:37:13.735323 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:13.735292 2564 generic.go:356] "Generic (PLEG): container finished" podID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerID="773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e" exitCode=0 Sep 10 18:37:13.735323 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:13.735315 2564 generic.go:356] "Generic (PLEG): container finished" podID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerID="55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818" exitCode=0 Sep 10 18:37:13.735323 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:13.735321 2564 generic.go:356] "Generic (PLEG): container finished" podID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerID="2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7" exitCode=0 Sep 10 18:37:13.735323 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:13.735327 2564 generic.go:356] "Generic (PLEG): container finished" podID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerID="60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945" exitCode=0 Sep 10 18:37:13.735610 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:13.735346 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8bb04f10-fe1b-4d07-bfed-890008ff5958","Type":"ContainerDied","Data":"773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e"} Sep 10 18:37:13.735610 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:13.735395 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8bb04f10-fe1b-4d07-bfed-890008ff5958","Type":"ContainerDied","Data":"55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818"} Sep 10 18:37:13.735610 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:13.735421 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8bb04f10-fe1b-4d07-bfed-890008ff5958","Type":"ContainerDied","Data":"2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7"} Sep 10 18:37:13.735610 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:13.735445 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8bb04f10-fe1b-4d07-bfed-890008ff5958","Type":"ContainerDied","Data":"60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945"} Sep 10 18:37:14.382628 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.382605 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.468943 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.468912 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb04f10-fe1b-4d07-bfed-890008ff5958-alertmanager-trusted-ca-bundle\") pod \"8bb04f10-fe1b-4d07-bfed-890008ff5958\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " Sep 10 18:37:14.469115 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.468953 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-config-volume\") pod \"8bb04f10-fe1b-4d07-bfed-890008ff5958\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " Sep 10 18:37:14.469115 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.468983 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy-web\") pod \"8bb04f10-fe1b-4d07-bfed-890008ff5958\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " Sep 10 18:37:14.469115 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.469007 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8bb04f10-fe1b-4d07-bfed-890008ff5958-tls-assets\") pod \"8bb04f10-fe1b-4d07-bfed-890008ff5958\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " Sep 10 18:37:14.469115 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.469027 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msqch\" (UniqueName: \"kubernetes.io/projected/8bb04f10-fe1b-4d07-bfed-890008ff5958-kube-api-access-msqch\") pod \"8bb04f10-fe1b-4d07-bfed-890008ff5958\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " Sep 10 18:37:14.469115 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.469047 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy\") pod \"8bb04f10-fe1b-4d07-bfed-890008ff5958\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " Sep 10 18:37:14.469115 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.469066 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8bb04f10-fe1b-4d07-bfed-890008ff5958-alertmanager-main-db\") pod \"8bb04f10-fe1b-4d07-bfed-890008ff5958\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " Sep 10 18:37:14.469115 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.469108 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-main-tls\") pod \"8bb04f10-fe1b-4d07-bfed-890008ff5958\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " Sep 10 18:37:14.469476 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.469132 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8bb04f10-fe1b-4d07-bfed-890008ff5958-config-out\") pod \"8bb04f10-fe1b-4d07-bfed-890008ff5958\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " Sep 10 18:37:14.469476 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.469164 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-cluster-tls-config\") pod \"8bb04f10-fe1b-4d07-bfed-890008ff5958\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " Sep 10 18:37:14.469476 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.469196 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bb04f10-fe1b-4d07-bfed-890008ff5958-metrics-client-ca\") pod \"8bb04f10-fe1b-4d07-bfed-890008ff5958\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " Sep 10 18:37:14.469476 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.469226 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-web-config\") pod \"8bb04f10-fe1b-4d07-bfed-890008ff5958\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " Sep 10 18:37:14.469476 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.469247 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy-metric\") pod \"8bb04f10-fe1b-4d07-bfed-890008ff5958\" (UID: \"8bb04f10-fe1b-4d07-bfed-890008ff5958\") " Sep 10 18:37:14.469792 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.469757 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bb04f10-fe1b-4d07-bfed-890008ff5958-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "8bb04f10-fe1b-4d07-bfed-890008ff5958" (UID: "8bb04f10-fe1b-4d07-bfed-890008ff5958"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:37:14.471929 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.471895 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bb04f10-fe1b-4d07-bfed-890008ff5958-kube-api-access-msqch" (OuterVolumeSpecName: "kube-api-access-msqch") pod "8bb04f10-fe1b-4d07-bfed-890008ff5958" (UID: "8bb04f10-fe1b-4d07-bfed-890008ff5958"). InnerVolumeSpecName "kube-api-access-msqch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:37:14.472042 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.471926 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bb04f10-fe1b-4d07-bfed-890008ff5958-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8bb04f10-fe1b-4d07-bfed-890008ff5958" (UID: "8bb04f10-fe1b-4d07-bfed-890008ff5958"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:37:14.472353 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.472259 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bb04f10-fe1b-4d07-bfed-890008ff5958-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "8bb04f10-fe1b-4d07-bfed-890008ff5958" (UID: "8bb04f10-fe1b-4d07-bfed-890008ff5958"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:37:14.472455 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.472414 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "8bb04f10-fe1b-4d07-bfed-890008ff5958" (UID: "8bb04f10-fe1b-4d07-bfed-890008ff5958"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:37:14.472752 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.472687 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "8bb04f10-fe1b-4d07-bfed-890008ff5958" (UID: "8bb04f10-fe1b-4d07-bfed-890008ff5958"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:37:14.472752 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.472712 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-config-volume" (OuterVolumeSpecName: "config-volume") pod "8bb04f10-fe1b-4d07-bfed-890008ff5958" (UID: "8bb04f10-fe1b-4d07-bfed-890008ff5958"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:37:14.472912 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.472776 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bb04f10-fe1b-4d07-bfed-890008ff5958-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "8bb04f10-fe1b-4d07-bfed-890008ff5958" (UID: "8bb04f10-fe1b-4d07-bfed-890008ff5958"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:37:14.473101 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.473067 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "8bb04f10-fe1b-4d07-bfed-890008ff5958" (UID: "8bb04f10-fe1b-4d07-bfed-890008ff5958"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:37:14.473199 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.473114 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "8bb04f10-fe1b-4d07-bfed-890008ff5958" (UID: "8bb04f10-fe1b-4d07-bfed-890008ff5958"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:37:14.473939 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.473918 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bb04f10-fe1b-4d07-bfed-890008ff5958-config-out" (OuterVolumeSpecName: "config-out") pod "8bb04f10-fe1b-4d07-bfed-890008ff5958" (UID: "8bb04f10-fe1b-4d07-bfed-890008ff5958"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:37:14.476577 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.476552 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "8bb04f10-fe1b-4d07-bfed-890008ff5958" (UID: "8bb04f10-fe1b-4d07-bfed-890008ff5958"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:37:14.485219 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.485199 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-web-config" (OuterVolumeSpecName: "web-config") pod "8bb04f10-fe1b-4d07-bfed-890008ff5958" (UID: "8bb04f10-fe1b-4d07-bfed-890008ff5958"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:37:14.570578 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.570544 2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8bb04f10-fe1b-4d07-bfed-890008ff5958-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:14.570578 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.570574 2564 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-config-volume\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:14.570578 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.570585 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:14.570782 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.570594 2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8bb04f10-fe1b-4d07-bfed-890008ff5958-tls-assets\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:14.570782 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.570603 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msqch\" (UniqueName: \"kubernetes.io/projected/8bb04f10-fe1b-4d07-bfed-890008ff5958-kube-api-access-msqch\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:14.570782 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.570611 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:14.570782 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.570620 2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8bb04f10-fe1b-4d07-bfed-890008ff5958-alertmanager-main-db\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:14.570782 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.570628 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-main-tls\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:14.570782 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.570635 2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8bb04f10-fe1b-4d07-bfed-890008ff5958-config-out\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:14.570782 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.570643 2564 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-cluster-tls-config\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:14.570782 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.570652 2564 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8bb04f10-fe1b-4d07-bfed-890008ff5958-metrics-client-ca\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:14.570782 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.570659 2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-web-config\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:14.570782 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.570667 2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8bb04f10-fe1b-4d07-bfed-890008ff5958-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:14.742519 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.742430 2564 generic.go:356] "Generic (PLEG): container finished" podID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerID="9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490" exitCode=0 Sep 10 18:37:14.742519 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.742454 2564 generic.go:356] "Generic (PLEG): container finished" podID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerID="73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb" exitCode=0 Sep 10 18:37:14.742677 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.742512 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8bb04f10-fe1b-4d07-bfed-890008ff5958","Type":"ContainerDied","Data":"9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490"} Sep 10 18:37:14.742677 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.742535 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.742677 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.742586 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8bb04f10-fe1b-4d07-bfed-890008ff5958","Type":"ContainerDied","Data":"73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb"} Sep 10 18:37:14.742677 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.742616 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8bb04f10-fe1b-4d07-bfed-890008ff5958","Type":"ContainerDied","Data":"9e5cb75236928925568799cb6879e89273be7ce4021e14ca262854c08d678376"} Sep 10 18:37:14.742677 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.742645 2564 scope.go:117] "RemoveContainer" containerID="773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e" Sep 10 18:37:14.750611 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.750538 2564 scope.go:117] "RemoveContainer" containerID="9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490" Sep 10 18:37:14.757088 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.757070 2564 scope.go:117] "RemoveContainer" containerID="55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818" Sep 10 18:37:14.763850 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.763834 2564 scope.go:117] "RemoveContainer" containerID="73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb" Sep 10 18:37:14.764396 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.764371 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 18:37:14.768225 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.768204 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 18:37:14.770965 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.770947 2564 scope.go:117] "RemoveContainer" containerID="2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7" Sep 10 18:37:14.777719 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.777698 2564 scope.go:117] "RemoveContainer" containerID="60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945" Sep 10 18:37:14.784300 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.784270 2564 scope.go:117] "RemoveContainer" containerID="0d3585d1a63cf7fdf41b4160a9cb52b9c0b210d0b0f9cfc2994b8eb8d3f1777c" Sep 10 18:37:14.791270 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.791253 2564 scope.go:117] "RemoveContainer" containerID="773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e" Sep 10 18:37:14.791562 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:37:14.791542 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e\": container with ID starting with 773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e not found: ID does not exist" containerID="773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e" Sep 10 18:37:14.791606 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.791570 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e"} err="failed to get container status \"773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e\": rpc error: code = NotFound desc = could not find container \"773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e\": container with ID starting with 773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e not found: ID does not exist" Sep 10 18:37:14.791606 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.791601 2564 scope.go:117] "RemoveContainer" containerID="9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490" Sep 10 18:37:14.791893 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:37:14.791868 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490\": container with ID starting with 9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490 not found: ID does not exist" containerID="9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490" Sep 10 18:37:14.791928 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.791905 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490"} err="failed to get container status \"9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490\": rpc error: code = NotFound desc = could not find container \"9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490\": container with ID starting with 9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490 not found: ID does not exist" Sep 10 18:37:14.791970 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.791927 2564 scope.go:117] "RemoveContainer" containerID="55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818" Sep 10 18:37:14.792467 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:37:14.792230 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818\": container with ID starting with 55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818 not found: ID does not exist" containerID="55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818" Sep 10 18:37:14.792467 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.792287 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818"} err="failed to get container status \"55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818\": rpc error: code = NotFound desc = could not find container \"55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818\": container with ID starting with 55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818 not found: ID does not exist" Sep 10 18:37:14.792467 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.792310 2564 scope.go:117] "RemoveContainer" containerID="73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb" Sep 10 18:37:14.792770 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:37:14.792645 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb\": container with ID starting with 73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb not found: ID does not exist" containerID="73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb" Sep 10 18:37:14.792770 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.792674 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb"} err="failed to get container status \"73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb\": rpc error: code = NotFound desc = could not find container \"73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb\": container with ID starting with 73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb not found: ID does not exist" Sep 10 18:37:14.792770 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.792695 2564 scope.go:117] "RemoveContainer" containerID="2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7" Sep 10 18:37:14.792964 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:37:14.792942 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7\": container with ID starting with 2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7 not found: ID does not exist" containerID="2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7" Sep 10 18:37:14.793012 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.792971 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7"} err="failed to get container status \"2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7\": rpc error: code = NotFound desc = could not find container \"2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7\": container with ID starting with 2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7 not found: ID does not exist" Sep 10 18:37:14.793012 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.792992 2564 scope.go:117] "RemoveContainer" containerID="60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945" Sep 10 18:37:14.793288 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:37:14.793270 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945\": container with ID starting with 60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945 not found: ID does not exist" containerID="60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945" Sep 10 18:37:14.793344 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.793291 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945"} err="failed to get container status \"60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945\": rpc error: code = NotFound desc = could not find container \"60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945\": container with ID starting with 60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945 not found: ID does not exist" Sep 10 18:37:14.793344 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.793306 2564 scope.go:117] "RemoveContainer" containerID="0d3585d1a63cf7fdf41b4160a9cb52b9c0b210d0b0f9cfc2994b8eb8d3f1777c" Sep 10 18:37:14.793672 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:37:14.793650 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d3585d1a63cf7fdf41b4160a9cb52b9c0b210d0b0f9cfc2994b8eb8d3f1777c\": container with ID starting with 0d3585d1a63cf7fdf41b4160a9cb52b9c0b210d0b0f9cfc2994b8eb8d3f1777c not found: ID does not exist" containerID="0d3585d1a63cf7fdf41b4160a9cb52b9c0b210d0b0f9cfc2994b8eb8d3f1777c" Sep 10 18:37:14.793738 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.793680 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d3585d1a63cf7fdf41b4160a9cb52b9c0b210d0b0f9cfc2994b8eb8d3f1777c"} err="failed to get container status \"0d3585d1a63cf7fdf41b4160a9cb52b9c0b210d0b0f9cfc2994b8eb8d3f1777c\": rpc error: code = NotFound desc = could not find container \"0d3585d1a63cf7fdf41b4160a9cb52b9c0b210d0b0f9cfc2994b8eb8d3f1777c\": container with ID starting with 0d3585d1a63cf7fdf41b4160a9cb52b9c0b210d0b0f9cfc2994b8eb8d3f1777c not found: ID does not exist" Sep 10 18:37:14.793738 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.793698 2564 scope.go:117] "RemoveContainer" containerID="773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e" Sep 10 18:37:14.793970 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.793951 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e"} err="failed to get container status \"773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e\": rpc error: code = NotFound desc = could not find container \"773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e\": container with ID starting with 773b8419b9a66a149b6188a8efe7c875796beeaf40b676572f7462f3813d653e not found: ID does not exist" Sep 10 18:37:14.794035 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.793971 2564 scope.go:117] "RemoveContainer" containerID="9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490" Sep 10 18:37:14.794222 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794204 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490"} err="failed to get container status \"9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490\": rpc error: code = NotFound desc = could not find container \"9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490\": container with ID starting with 9312c27fb4ee36d9347e34a9840b3590b06dee922d38bfeeb3c0bc26df160490 not found: ID does not exist" Sep 10 18:37:14.794267 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794227 2564 scope.go:117] "RemoveContainer" containerID="55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818" Sep 10 18:37:14.794299 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794209 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 18:37:14.794494 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794462 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818"} err="failed to get container status \"55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818\": rpc error: code = NotFound desc = could not find container \"55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818\": container with ID starting with 55aa7fb49002901bfb69d66782fba490349ffa0724e1c0634e326c494f4ee818 not found: ID does not exist" Sep 10 18:37:14.794538 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794500 2564 scope.go:117] "RemoveContainer" containerID="73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb" Sep 10 18:37:14.794596 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794581 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="kube-rbac-proxy-web" Sep 10 18:37:14.794648 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794600 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="kube-rbac-proxy-web" Sep 10 18:37:14.794648 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794612 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="kube-rbac-proxy" Sep 10 18:37:14.794648 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794622 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="kube-rbac-proxy" Sep 10 18:37:14.794648 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794641 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="prom-label-proxy" Sep 10 18:37:14.794781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794649 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="prom-label-proxy" Sep 10 18:37:14.794781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794679 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05dbe2c5-05cf-4b20-99e2-a50b46fe0252" containerName="registry" Sep 10 18:37:14.794781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794688 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="05dbe2c5-05cf-4b20-99e2-a50b46fe0252" containerName="registry" Sep 10 18:37:14.794781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794698 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="init-config-reloader" Sep 10 18:37:14.794781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794704 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="init-config-reloader" Sep 10 18:37:14.794781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794709 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="kube-rbac-proxy-metric" Sep 10 18:37:14.794781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794714 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="kube-rbac-proxy-metric" Sep 10 18:37:14.794781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794722 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="alertmanager" Sep 10 18:37:14.794781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794727 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="alertmanager" Sep 10 18:37:14.794781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794735 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="config-reloader" Sep 10 18:37:14.794781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794741 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="config-reloader" Sep 10 18:37:14.794781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794745 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb"} err="failed to get container status \"73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb\": rpc error: code = NotFound desc = could not find container \"73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb\": container with ID starting with 73756d48dd2e54746efe056fdac2024152cf6c898ce4c338f864421ce39b0cfb not found: ID does not exist" Sep 10 18:37:14.794781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794759 2564 scope.go:117] "RemoveContainer" containerID="2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7" Sep 10 18:37:14.795277 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794801 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="alertmanager" Sep 10 18:37:14.795277 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794810 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="kube-rbac-proxy-metric" Sep 10 18:37:14.795277 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794819 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="kube-rbac-proxy" Sep 10 18:37:14.795277 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794830 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="05dbe2c5-05cf-4b20-99e2-a50b46fe0252" containerName="registry" Sep 10 18:37:14.795277 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794836 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="config-reloader" Sep 10 18:37:14.795277 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794842 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="prom-label-proxy" Sep 10 18:37:14.795277 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.794848 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" containerName="kube-rbac-proxy-web" Sep 10 18:37:14.795277 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.795011 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7"} err="failed to get container status \"2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7\": rpc error: code = NotFound desc = could not find container \"2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7\": container with ID starting with 2c67cc7d46cc670390540a7f871860150fb82fff78b96c616ccf419de7ab3fc7 not found: ID does not exist" Sep 10 18:37:14.795277 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.795029 2564 scope.go:117] "RemoveContainer" containerID="60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945" Sep 10 18:37:14.795277 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.795271 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945"} err="failed to get container status \"60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945\": rpc error: code = NotFound desc = could not find container \"60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945\": container with ID starting with 60f83bc48d016c7abad4fc45fae4ae37dac57e6bcf2bb754d3bac890790e7945 not found: ID does not exist" Sep 10 18:37:14.795644 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.795288 2564 scope.go:117] "RemoveContainer" containerID="0d3585d1a63cf7fdf41b4160a9cb52b9c0b210d0b0f9cfc2994b8eb8d3f1777c" Sep 10 18:37:14.795644 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.795546 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d3585d1a63cf7fdf41b4160a9cb52b9c0b210d0b0f9cfc2994b8eb8d3f1777c"} err="failed to get container status \"0d3585d1a63cf7fdf41b4160a9cb52b9c0b210d0b0f9cfc2994b8eb8d3f1777c\": rpc error: code = NotFound desc = could not find container \"0d3585d1a63cf7fdf41b4160a9cb52b9c0b210d0b0f9cfc2994b8eb8d3f1777c\": container with ID starting with 0d3585d1a63cf7fdf41b4160a9cb52b9c0b210d0b0f9cfc2994b8eb8d3f1777c not found: ID does not exist" Sep 10 18:37:14.801672 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.801654 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.804017 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.803995 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 18:37:14.804119 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.803998 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 18:37:14.804119 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.804034 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 18:37:14.804119 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.804042 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 18:37:14.804119 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.804050 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rdzqz\"" Sep 10 18:37:14.804307 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.804194 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 18:37:14.804307 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.804251 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 18:37:14.804563 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.804547 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 18:37:14.804644 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.804603 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 18:37:14.809738 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.809710 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 18:37:14.811419 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.811393 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 18:37:14.872385 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.872350 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/afff2259-85d9-4d94-872e-da941e8fdc9a-config-out\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.872385 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.872379 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-web-config\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.872599 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.872427 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afff2259-85d9-4d94-872e-da941e8fdc9a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.872599 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.872472 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.872599 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.872523 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.872599 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.872545 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afff2259-85d9-4d94-872e-da941e8fdc9a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.872599 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.872562 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/afff2259-85d9-4d94-872e-da941e8fdc9a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.872599 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.872585 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.872777 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.872624 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-config-volume\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.872777 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.872642 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/afff2259-85d9-4d94-872e-da941e8fdc9a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.872777 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.872661 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.872777 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.872688 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.872777 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.872714 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-964cz\" (UniqueName: \"kubernetes.io/projected/afff2259-85d9-4d94-872e-da941e8fdc9a-kube-api-access-964cz\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.973418 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.973378 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afff2259-85d9-4d94-872e-da941e8fdc9a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.973418 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.973419 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/afff2259-85d9-4d94-872e-da941e8fdc9a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.973644 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.973443 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.973688 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.973665 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-config-volume\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.973729 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.973712 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/afff2259-85d9-4d94-872e-da941e8fdc9a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.973783 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.973748 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.973836 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.973788 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.973947 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.973918 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-964cz\" (UniqueName: \"kubernetes.io/projected/afff2259-85d9-4d94-872e-da941e8fdc9a-kube-api-access-964cz\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.974016 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.973997 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/afff2259-85d9-4d94-872e-da941e8fdc9a-config-out\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.974071 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.974023 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-web-config\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.974071 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.974065 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afff2259-85d9-4d94-872e-da941e8fdc9a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.974206 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.974076 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/afff2259-85d9-4d94-872e-da941e8fdc9a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.974206 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.974097 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.974206 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.974134 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.974354 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.974252 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afff2259-85d9-4d94-872e-da941e8fdc9a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.975245 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.975215 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afff2259-85d9-4d94-872e-da941e8fdc9a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.976678 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.976652 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-config-volume\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.976678 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.976672 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.976975 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.976953 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/afff2259-85d9-4d94-872e-da941e8fdc9a-config-out\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.977129 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.977103 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.977244 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.977103 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.977244 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.977201 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.977244 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.977213 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.977503 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.977463 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/afff2259-85d9-4d94-872e-da941e8fdc9a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.978334 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.978318 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/afff2259-85d9-4d94-872e-da941e8fdc9a-web-config\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:14.981818 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:14.981796 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-964cz\" (UniqueName: \"kubernetes.io/projected/afff2259-85d9-4d94-872e-da941e8fdc9a-kube-api-access-964cz\") pod \"alertmanager-main-0\" (UID: \"afff2259-85d9-4d94-872e-da941e8fdc9a\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:15.112292 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:15.112207 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 18:37:15.390252 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:15.390224 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 18:37:15.392170 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:37:15.392140 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafff2259_85d9_4d94_872e_da941e8fdc9a.slice/crio-5db1259853cdc61a460556ef0a3106bcddbf800926fe7a60bf30e0cd58a5ff29 WatchSource:0}: Error finding container 5db1259853cdc61a460556ef0a3106bcddbf800926fe7a60bf30e0cd58a5ff29: Status 404 returned error can't find the container with id 5db1259853cdc61a460556ef0a3106bcddbf800926fe7a60bf30e0cd58a5ff29 Sep 10 18:37:15.447389 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:15.447360 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bb04f10-fe1b-4d07-bfed-890008ff5958" path="/var/lib/kubelet/pods/8bb04f10-fe1b-4d07-bfed-890008ff5958/volumes" Sep 10 18:37:15.747903 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:15.747868 2564 generic.go:356] "Generic (PLEG): container finished" podID="afff2259-85d9-4d94-872e-da941e8fdc9a" containerID="a548ecf41140c8666fd1ee159fc4178d1baf073428a5d3e113f088891522d2f9" exitCode=0 Sep 10 18:37:15.748076 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:15.747954 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afff2259-85d9-4d94-872e-da941e8fdc9a","Type":"ContainerDied","Data":"a548ecf41140c8666fd1ee159fc4178d1baf073428a5d3e113f088891522d2f9"} Sep 10 18:37:15.748076 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:15.748033 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afff2259-85d9-4d94-872e-da941e8fdc9a","Type":"ContainerStarted","Data":"5db1259853cdc61a460556ef0a3106bcddbf800926fe7a60bf30e0cd58a5ff29"} Sep 10 18:37:16.755746 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:16.755709 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afff2259-85d9-4d94-872e-da941e8fdc9a","Type":"ContainerStarted","Data":"f0bf70b9ba299da6923386d1ba7134157a0c2337d3895666f7881b8d507f9210"} Sep 10 18:37:16.756126 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:16.755824 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afff2259-85d9-4d94-872e-da941e8fdc9a","Type":"ContainerStarted","Data":"51eedaad99a0a3b7f33fad262e7ecc24e030ffd35ff14f0d615ce4c7589a9957"} Sep 10 18:37:16.756126 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:16.755869 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afff2259-85d9-4d94-872e-da941e8fdc9a","Type":"ContainerStarted","Data":"34fcd258088bfa84fd3a8b7981d6483afc3619b051fcb2bf79e7e4d22fb273a1"} Sep 10 18:37:16.756126 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:16.755909 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afff2259-85d9-4d94-872e-da941e8fdc9a","Type":"ContainerStarted","Data":"ad55aef88e9d09c64bf2c1b80771b8ed3537ba0b5f050bd46c04262746d7ac56"} Sep 10 18:37:16.756126 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:16.755945 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afff2259-85d9-4d94-872e-da941e8fdc9a","Type":"ContainerStarted","Data":"04eb479f48c756791bfc80ada66074a844aa2fb41065cdb89f785cc576419487"} Sep 10 18:37:16.756126 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:16.755981 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afff2259-85d9-4d94-872e-da941e8fdc9a","Type":"ContainerStarted","Data":"f0bfebeee8aa8293037e27a911e98449bb67165e1f7d5cd34a11e1b362a4e4c3"} Sep 10 18:37:16.780708 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:16.780657 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.76306998 podStartE2EDuration="2.780641603s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.571623ms" isStatefulPod=false podCreationTimestamp="2026-09-10 18:37:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:37:16.778089369 +0000 UTC m=+365.921392613" watchObservedRunningTime="2026-09-10 18:37:16.780641603 +0000 UTC m=+365.923944847" Sep 10 18:37:17.197202 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.197103 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c"] Sep 10 18:37:17.201408 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.201380 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.203593 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.203431 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 10 18:37:17.203722 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.203599 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 10 18:37:17.203809 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.203787 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 10 18:37:17.204320 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.204121 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 10 18:37:17.204320 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.204166 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-tvkpl\"" Sep 10 18:37:17.204320 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.204243 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 10 18:37:17.208786 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.208765 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 10 18:37:17.215794 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.215771 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c"] Sep 10 18:37:17.294906 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.294871 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/73e86922-fc64-445d-a85f-0f0c4b849ac2-federate-client-tls\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.294906 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.294913 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73e86922-fc64-445d-a85f-0f0c4b849ac2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.295135 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.294944 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73e86922-fc64-445d-a85f-0f0c4b849ac2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.295135 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.295074 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/73e86922-fc64-445d-a85f-0f0c4b849ac2-secret-telemeter-client\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.295135 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.295114 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73e86922-fc64-445d-a85f-0f0c4b849ac2-metrics-client-ca\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.295271 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.295156 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4gxf\" (UniqueName: \"kubernetes.io/projected/73e86922-fc64-445d-a85f-0f0c4b849ac2-kube-api-access-d4gxf\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.295271 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.295197 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/73e86922-fc64-445d-a85f-0f0c4b849ac2-telemeter-client-tls\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.295271 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.295231 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73e86922-fc64-445d-a85f-0f0c4b849ac2-serving-certs-ca-bundle\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.395939 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.395892 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73e86922-fc64-445d-a85f-0f0c4b849ac2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.396111 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.395989 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/73e86922-fc64-445d-a85f-0f0c4b849ac2-secret-telemeter-client\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.396111 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.396027 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73e86922-fc64-445d-a85f-0f0c4b849ac2-metrics-client-ca\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.396111 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.396060 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4gxf\" (UniqueName: \"kubernetes.io/projected/73e86922-fc64-445d-a85f-0f0c4b849ac2-kube-api-access-d4gxf\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.396111 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.396100 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/73e86922-fc64-445d-a85f-0f0c4b849ac2-telemeter-client-tls\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.396324 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.396128 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73e86922-fc64-445d-a85f-0f0c4b849ac2-serving-certs-ca-bundle\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.396324 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.396182 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/73e86922-fc64-445d-a85f-0f0c4b849ac2-federate-client-tls\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.396324 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.396207 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73e86922-fc64-445d-a85f-0f0c4b849ac2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.397048 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.396989 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73e86922-fc64-445d-a85f-0f0c4b849ac2-metrics-client-ca\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.397228 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.397053 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73e86922-fc64-445d-a85f-0f0c4b849ac2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.397228 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.397142 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73e86922-fc64-445d-a85f-0f0c4b849ac2-serving-certs-ca-bundle\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.398920 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.398895 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73e86922-fc64-445d-a85f-0f0c4b849ac2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.399194 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.399165 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/73e86922-fc64-445d-a85f-0f0c4b849ac2-telemeter-client-tls\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.399289 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.399165 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/73e86922-fc64-445d-a85f-0f0c4b849ac2-secret-telemeter-client\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.401364 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.401341 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/73e86922-fc64-445d-a85f-0f0c4b849ac2-federate-client-tls\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.403996 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.403974 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4gxf\" (UniqueName: \"kubernetes.io/projected/73e86922-fc64-445d-a85f-0f0c4b849ac2-kube-api-access-d4gxf\") pod \"telemeter-client-7f59bbb4cb-wvk9c\" (UID: \"73e86922-fc64-445d-a85f-0f0c4b849ac2\") " pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.493000 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.492960 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 18:37:17.493341 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.493306 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="prometheus" containerID="cri-o://1c34018c2e065dc17b103cef43c7c18995e35e7891ad5a91f0748768691972d5" gracePeriod=600 Sep 10 18:37:17.493425 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.493357 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="kube-rbac-proxy-thanos" containerID="cri-o://6eac00504c049c61218406ddec8af210b6b6b0886ec2349e13c920710c27281b" gracePeriod=600 Sep 10 18:37:17.493425 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.493383 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="thanos-sidecar" containerID="cri-o://877ab1e4af5d644a113a380d16d3cc07804399eba8089562f9032b13431c210b" gracePeriod=600 Sep 10 18:37:17.493573 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.493388 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="config-reloader" containerID="cri-o://3488f8c1468076f34ab5559e3a428f052f3ef62269d89808d108e9021749227c" gracePeriod=600 Sep 10 18:37:17.493636 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.493411 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="kube-rbac-proxy-web" containerID="cri-o://9aebe97131ae59cb34866a74300421d058c69e1426400234a2f7242e3961078b" gracePeriod=600 Sep 10 18:37:17.493740 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.493695 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="kube-rbac-proxy" containerID="cri-o://22dbe8b6b490a1e8f57c31e7c3a6e4e8bd505e6651fc9ab62513e616e102ea48" gracePeriod=600 Sep 10 18:37:17.516383 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.516358 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" Sep 10 18:37:17.764180 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.764091 2564 generic.go:356] "Generic (PLEG): container finished" podID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerID="6eac00504c049c61218406ddec8af210b6b6b0886ec2349e13c920710c27281b" exitCode=0 Sep 10 18:37:17.764180 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.764116 2564 generic.go:356] "Generic (PLEG): container finished" podID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerID="22dbe8b6b490a1e8f57c31e7c3a6e4e8bd505e6651fc9ab62513e616e102ea48" exitCode=0 Sep 10 18:37:17.764180 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.764122 2564 generic.go:356] "Generic (PLEG): container finished" podID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerID="877ab1e4af5d644a113a380d16d3cc07804399eba8089562f9032b13431c210b" exitCode=0 Sep 10 18:37:17.764180 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.764128 2564 generic.go:356] "Generic (PLEG): container finished" podID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerID="3488f8c1468076f34ab5559e3a428f052f3ef62269d89808d108e9021749227c" exitCode=0 Sep 10 18:37:17.764180 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.764134 2564 generic.go:356] "Generic (PLEG): container finished" podID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerID="1c34018c2e065dc17b103cef43c7c18995e35e7891ad5a91f0748768691972d5" exitCode=0 Sep 10 18:37:17.764180 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.764160 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4541aa2-c3d6-4af7-bd34-de202b1375b2","Type":"ContainerDied","Data":"6eac00504c049c61218406ddec8af210b6b6b0886ec2349e13c920710c27281b"} Sep 10 18:37:17.764867 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.764268 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4541aa2-c3d6-4af7-bd34-de202b1375b2","Type":"ContainerDied","Data":"22dbe8b6b490a1e8f57c31e7c3a6e4e8bd505e6651fc9ab62513e616e102ea48"} Sep 10 18:37:17.764867 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.764311 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4541aa2-c3d6-4af7-bd34-de202b1375b2","Type":"ContainerDied","Data":"877ab1e4af5d644a113a380d16d3cc07804399eba8089562f9032b13431c210b"} Sep 10 18:37:17.764867 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.764348 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4541aa2-c3d6-4af7-bd34-de202b1375b2","Type":"ContainerDied","Data":"3488f8c1468076f34ab5559e3a428f052f3ef62269d89808d108e9021749227c"} Sep 10 18:37:17.764867 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.764386 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4541aa2-c3d6-4af7-bd34-de202b1375b2","Type":"ContainerDied","Data":"1c34018c2e065dc17b103cef43c7c18995e35e7891ad5a91f0748768691972d5"} Sep 10 18:37:17.811324 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.811289 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c"] Sep 10 18:37:17.814506 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:37:17.814455 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73e86922_fc64_445d_a85f_0f0c4b849ac2.slice/crio-27829f76fc487fa5424dd06c94f5106d485deaadcd45ae547ba86ede22b8d52c WatchSource:0}: Error finding container 27829f76fc487fa5424dd06c94f5106d485deaadcd45ae547ba86ede22b8d52c: Status 404 returned error can't find the container with id 27829f76fc487fa5424dd06c94f5106d485deaadcd45ae547ba86ede22b8d52c Sep 10 18:37:17.816433 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:17.816416 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 18:37:18.745698 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.745668 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:18.771474 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.771437 2564 generic.go:356] "Generic (PLEG): container finished" podID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerID="9aebe97131ae59cb34866a74300421d058c69e1426400234a2f7242e3961078b" exitCode=0 Sep 10 18:37:18.771874 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.771514 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4541aa2-c3d6-4af7-bd34-de202b1375b2","Type":"ContainerDied","Data":"9aebe97131ae59cb34866a74300421d058c69e1426400234a2f7242e3961078b"} Sep 10 18:37:18.771874 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.771553 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:18.771874 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.771637 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a4541aa2-c3d6-4af7-bd34-de202b1375b2","Type":"ContainerDied","Data":"8094bcad3cd8f33296eb4b5c8fdb23fa895a5778ed323c261820c68bcae83f26"} Sep 10 18:37:18.771874 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.771688 2564 scope.go:117] "RemoveContainer" containerID="6eac00504c049c61218406ddec8af210b6b6b0886ec2349e13c920710c27281b" Sep 10 18:37:18.772742 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.772658 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" event={"ID":"73e86922-fc64-445d-a85f-0f0c4b849ac2","Type":"ContainerStarted","Data":"27829f76fc487fa5424dd06c94f5106d485deaadcd45ae547ba86ede22b8d52c"} Sep 10 18:37:18.780204 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.780185 2564 scope.go:117] "RemoveContainer" containerID="22dbe8b6b490a1e8f57c31e7c3a6e4e8bd505e6651fc9ab62513e616e102ea48" Sep 10 18:37:18.789265 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.789240 2564 scope.go:117] "RemoveContainer" containerID="9aebe97131ae59cb34866a74300421d058c69e1426400234a2f7242e3961078b" Sep 10 18:37:18.799141 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.799116 2564 scope.go:117] "RemoveContainer" containerID="877ab1e4af5d644a113a380d16d3cc07804399eba8089562f9032b13431c210b" Sep 10 18:37:18.807148 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807123 2564 scope.go:117] "RemoveContainer" containerID="3488f8c1468076f34ab5559e3a428f052f3ef62269d89808d108e9021749227c" Sep 10 18:37:18.807330 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807292 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-db\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.807425 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807347 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.807425 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807381 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6b29t\" (UniqueName: \"kubernetes.io/projected/a4541aa2-c3d6-4af7-bd34-de202b1375b2-kube-api-access-6b29t\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.807425 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807414 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-thanos-prometheus-http-client-file\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.807645 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807627 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-trusted-ca-bundle\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.807698 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807663 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-web-config\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.807751 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807711 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4541aa2-c3d6-4af7-bd34-de202b1375b2-tls-assets\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.807800 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807747 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-tls\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.807800 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807771 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-0\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.807902 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807805 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-config\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.807902 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807831 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4541aa2-c3d6-4af7-bd34-de202b1375b2-config-out\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.807902 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807860 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.807902 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807896 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-kubelet-serving-ca-bundle\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.808562 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807932 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-1\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.808562 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807973 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-serving-certs-ca-bundle\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.808562 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.807997 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-grpc-tls\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.808562 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.808010 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:37:18.808562 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.808024 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-2\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.808562 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.808061 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-metrics-client-ca\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.808562 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.808101 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-kube-rbac-proxy\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.808562 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.808158 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-metrics-client-certs\") pod \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\" (UID: \"a4541aa2-c3d6-4af7-bd34-de202b1375b2\") " Sep 10 18:37:18.808562 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.808519 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:37:18.809017 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.808814 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:37:18.809017 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.808885 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:37:18.809017 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.808923 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:37:18.809248 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.809223 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:37:18.809654 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.809429 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.809654 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.809454 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-metrics-client-ca\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.809654 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.809470 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-db\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.809654 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.809501 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.809654 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.809522 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.809654 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.809539 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.811803 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.811513 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4541aa2-c3d6-4af7-bd34-de202b1375b2-kube-api-access-6b29t" (OuterVolumeSpecName: "kube-api-access-6b29t") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "kube-api-access-6b29t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:37:18.811803 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.811596 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:37:18.811952 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.811827 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4541aa2-c3d6-4af7-bd34-de202b1375b2-config-out" (OuterVolumeSpecName: "config-out") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 18:37:18.812754 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.812056 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:37:18.812754 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.812501 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-config" (OuterVolumeSpecName: "config") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:37:18.813419 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.813026 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:37:18.813419 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.813290 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 18:37:18.813419 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.813350 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:37:18.813419 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.813389 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:37:18.813709 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.813464 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4541aa2-c3d6-4af7-bd34-de202b1375b2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:37:18.815012 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.814954 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:37:18.815112 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.815058 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:37:18.815470 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.815436 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:37:18.825818 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.825794 2564 scope.go:117] "RemoveContainer" containerID="1c34018c2e065dc17b103cef43c7c18995e35e7891ad5a91f0748768691972d5" Sep 10 18:37:18.826841 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.826797 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-web-config" (OuterVolumeSpecName: "web-config") pod "a4541aa2-c3d6-4af7-bd34-de202b1375b2" (UID: "a4541aa2-c3d6-4af7-bd34-de202b1375b2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 18:37:18.834292 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.834270 2564 scope.go:117] "RemoveContainer" containerID="c4cf59b4392bf21b7eec9b4b84b68cf594940aeeef1e8e3f427ceaae324ee0e0" Sep 10 18:37:18.841748 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.841717 2564 scope.go:117] "RemoveContainer" containerID="6eac00504c049c61218406ddec8af210b6b6b0886ec2349e13c920710c27281b" Sep 10 18:37:18.842041 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:37:18.842016 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eac00504c049c61218406ddec8af210b6b6b0886ec2349e13c920710c27281b\": container with ID starting with 6eac00504c049c61218406ddec8af210b6b6b0886ec2349e13c920710c27281b not found: ID does not exist" containerID="6eac00504c049c61218406ddec8af210b6b6b0886ec2349e13c920710c27281b" Sep 10 18:37:18.842128 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.842049 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eac00504c049c61218406ddec8af210b6b6b0886ec2349e13c920710c27281b"} err="failed to get container status \"6eac00504c049c61218406ddec8af210b6b6b0886ec2349e13c920710c27281b\": rpc error: code = NotFound desc = could not find container \"6eac00504c049c61218406ddec8af210b6b6b0886ec2349e13c920710c27281b\": container with ID starting with 6eac00504c049c61218406ddec8af210b6b6b0886ec2349e13c920710c27281b not found: ID does not exist" Sep 10 18:37:18.842128 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.842070 2564 scope.go:117] "RemoveContainer" containerID="22dbe8b6b490a1e8f57c31e7c3a6e4e8bd505e6651fc9ab62513e616e102ea48" Sep 10 18:37:18.842353 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:37:18.842325 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22dbe8b6b490a1e8f57c31e7c3a6e4e8bd505e6651fc9ab62513e616e102ea48\": container with ID starting with 22dbe8b6b490a1e8f57c31e7c3a6e4e8bd505e6651fc9ab62513e616e102ea48 not found: ID does not exist" containerID="22dbe8b6b490a1e8f57c31e7c3a6e4e8bd505e6651fc9ab62513e616e102ea48" Sep 10 18:37:18.842422 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.842361 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22dbe8b6b490a1e8f57c31e7c3a6e4e8bd505e6651fc9ab62513e616e102ea48"} err="failed to get container status \"22dbe8b6b490a1e8f57c31e7c3a6e4e8bd505e6651fc9ab62513e616e102ea48\": rpc error: code = NotFound desc = could not find container \"22dbe8b6b490a1e8f57c31e7c3a6e4e8bd505e6651fc9ab62513e616e102ea48\": container with ID starting with 22dbe8b6b490a1e8f57c31e7c3a6e4e8bd505e6651fc9ab62513e616e102ea48 not found: ID does not exist" Sep 10 18:37:18.842422 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.842379 2564 scope.go:117] "RemoveContainer" containerID="9aebe97131ae59cb34866a74300421d058c69e1426400234a2f7242e3961078b" Sep 10 18:37:18.842711 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:37:18.842688 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9aebe97131ae59cb34866a74300421d058c69e1426400234a2f7242e3961078b\": container with ID starting with 9aebe97131ae59cb34866a74300421d058c69e1426400234a2f7242e3961078b not found: ID does not exist" containerID="9aebe97131ae59cb34866a74300421d058c69e1426400234a2f7242e3961078b" Sep 10 18:37:18.842782 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.842716 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9aebe97131ae59cb34866a74300421d058c69e1426400234a2f7242e3961078b"} err="failed to get container status \"9aebe97131ae59cb34866a74300421d058c69e1426400234a2f7242e3961078b\": rpc error: code = NotFound desc = could not find container \"9aebe97131ae59cb34866a74300421d058c69e1426400234a2f7242e3961078b\": container with ID starting with 9aebe97131ae59cb34866a74300421d058c69e1426400234a2f7242e3961078b not found: ID does not exist" Sep 10 18:37:18.842782 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.842737 2564 scope.go:117] "RemoveContainer" containerID="877ab1e4af5d644a113a380d16d3cc07804399eba8089562f9032b13431c210b" Sep 10 18:37:18.843014 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:37:18.842995 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"877ab1e4af5d644a113a380d16d3cc07804399eba8089562f9032b13431c210b\": container with ID starting with 877ab1e4af5d644a113a380d16d3cc07804399eba8089562f9032b13431c210b not found: ID does not exist" containerID="877ab1e4af5d644a113a380d16d3cc07804399eba8089562f9032b13431c210b" Sep 10 18:37:18.843075 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.843015 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"877ab1e4af5d644a113a380d16d3cc07804399eba8089562f9032b13431c210b"} err="failed to get container status \"877ab1e4af5d644a113a380d16d3cc07804399eba8089562f9032b13431c210b\": rpc error: code = NotFound desc = could not find container \"877ab1e4af5d644a113a380d16d3cc07804399eba8089562f9032b13431c210b\": container with ID starting with 877ab1e4af5d644a113a380d16d3cc07804399eba8089562f9032b13431c210b not found: ID does not exist" Sep 10 18:37:18.843075 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.843029 2564 scope.go:117] "RemoveContainer" containerID="3488f8c1468076f34ab5559e3a428f052f3ef62269d89808d108e9021749227c" Sep 10 18:37:18.843283 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:37:18.843266 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3488f8c1468076f34ab5559e3a428f052f3ef62269d89808d108e9021749227c\": container with ID starting with 3488f8c1468076f34ab5559e3a428f052f3ef62269d89808d108e9021749227c not found: ID does not exist" containerID="3488f8c1468076f34ab5559e3a428f052f3ef62269d89808d108e9021749227c" Sep 10 18:37:18.843345 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.843284 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3488f8c1468076f34ab5559e3a428f052f3ef62269d89808d108e9021749227c"} err="failed to get container status \"3488f8c1468076f34ab5559e3a428f052f3ef62269d89808d108e9021749227c\": rpc error: code = NotFound desc = could not find container \"3488f8c1468076f34ab5559e3a428f052f3ef62269d89808d108e9021749227c\": container with ID starting with 3488f8c1468076f34ab5559e3a428f052f3ef62269d89808d108e9021749227c not found: ID does not exist" Sep 10 18:37:18.843345 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.843296 2564 scope.go:117] "RemoveContainer" containerID="1c34018c2e065dc17b103cef43c7c18995e35e7891ad5a91f0748768691972d5" Sep 10 18:37:18.843569 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:37:18.843551 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c34018c2e065dc17b103cef43c7c18995e35e7891ad5a91f0748768691972d5\": container with ID starting with 1c34018c2e065dc17b103cef43c7c18995e35e7891ad5a91f0748768691972d5 not found: ID does not exist" containerID="1c34018c2e065dc17b103cef43c7c18995e35e7891ad5a91f0748768691972d5" Sep 10 18:37:18.843630 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.843575 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c34018c2e065dc17b103cef43c7c18995e35e7891ad5a91f0748768691972d5"} err="failed to get container status \"1c34018c2e065dc17b103cef43c7c18995e35e7891ad5a91f0748768691972d5\": rpc error: code = NotFound desc = could not find container \"1c34018c2e065dc17b103cef43c7c18995e35e7891ad5a91f0748768691972d5\": container with ID starting with 1c34018c2e065dc17b103cef43c7c18995e35e7891ad5a91f0748768691972d5 not found: ID does not exist" Sep 10 18:37:18.843630 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.843593 2564 scope.go:117] "RemoveContainer" containerID="c4cf59b4392bf21b7eec9b4b84b68cf594940aeeef1e8e3f427ceaae324ee0e0" Sep 10 18:37:18.843819 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:37:18.843802 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4cf59b4392bf21b7eec9b4b84b68cf594940aeeef1e8e3f427ceaae324ee0e0\": container with ID starting with c4cf59b4392bf21b7eec9b4b84b68cf594940aeeef1e8e3f427ceaae324ee0e0 not found: ID does not exist" containerID="c4cf59b4392bf21b7eec9b4b84b68cf594940aeeef1e8e3f427ceaae324ee0e0" Sep 10 18:37:18.843881 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.843828 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4cf59b4392bf21b7eec9b4b84b68cf594940aeeef1e8e3f427ceaae324ee0e0"} err="failed to get container status \"c4cf59b4392bf21b7eec9b4b84b68cf594940aeeef1e8e3f427ceaae324ee0e0\": rpc error: code = NotFound desc = could not find container \"c4cf59b4392bf21b7eec9b4b84b68cf594940aeeef1e8e3f427ceaae324ee0e0\": container with ID starting with c4cf59b4392bf21b7eec9b4b84b68cf594940aeeef1e8e3f427ceaae324ee0e0 not found: ID does not exist" Sep 10 18:37:18.910544 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.910510 2564 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-kube-rbac-proxy\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.910544 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.910543 2564 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-metrics-client-certs\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.910715 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.910559 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.910715 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.910574 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6b29t\" (UniqueName: \"kubernetes.io/projected/a4541aa2-c3d6-4af7-bd34-de202b1375b2-kube-api-access-6b29t\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.910715 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.910588 2564 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.910715 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.910602 2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-web-config\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.910715 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.910615 2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4541aa2-c3d6-4af7-bd34-de202b1375b2-tls-assets\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.910715 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.910628 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.910715 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.910642 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.910715 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.910656 2564 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-config\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.910715 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.910668 2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4541aa2-c3d6-4af7-bd34-de202b1375b2-config-out\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.910715 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.910682 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.910715 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.910695 2564 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a4541aa2-c3d6-4af7-bd34-de202b1375b2-secret-grpc-tls\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:18.910715 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:18.910708 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a4541aa2-c3d6-4af7-bd34-de202b1375b2-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:37:19.096119 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.096092 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 18:37:19.100084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.100061 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 18:37:19.127705 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.127683 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 18:37:19.127966 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.127955 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="kube-rbac-proxy" Sep 10 18:37:19.128010 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.127968 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="kube-rbac-proxy" Sep 10 18:37:19.128010 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.127977 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="kube-rbac-proxy-web" Sep 10 18:37:19.128010 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.127983 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="kube-rbac-proxy-web" Sep 10 18:37:19.128010 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.127992 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="thanos-sidecar" Sep 10 18:37:19.128010 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.127997 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="thanos-sidecar" Sep 10 18:37:19.128149 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.128014 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="config-reloader" Sep 10 18:37:19.128149 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.128020 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="config-reloader" Sep 10 18:37:19.128149 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.128028 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="init-config-reloader" Sep 10 18:37:19.128149 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.128033 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="init-config-reloader" Sep 10 18:37:19.128149 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.128039 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="prometheus" Sep 10 18:37:19.128149 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.128044 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="prometheus" Sep 10 18:37:19.128149 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.128051 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="kube-rbac-proxy-thanos" Sep 10 18:37:19.128149 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.128057 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="kube-rbac-proxy-thanos" Sep 10 18:37:19.128149 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.128106 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="kube-rbac-proxy-web" Sep 10 18:37:19.128149 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.128115 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="config-reloader" Sep 10 18:37:19.128149 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.128122 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="thanos-sidecar" Sep 10 18:37:19.128149 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.128128 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="kube-rbac-proxy" Sep 10 18:37:19.128149 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.128133 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="kube-rbac-proxy-thanos" Sep 10 18:37:19.128149 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.128139 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" containerName="prometheus" Sep 10 18:37:19.134581 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.134560 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.137067 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.137046 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 18:37:19.137171 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.137159 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 18:37:19.137231 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.137159 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 18:37:19.137303 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.137269 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-fldfc\"" Sep 10 18:37:19.137388 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.137370 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 18:37:19.137912 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.137360 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 18:37:19.137912 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.137413 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 18:37:19.137912 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.137440 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 18:37:19.137912 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.137468 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 18:37:19.137912 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.137705 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bnfnenbtu9m44\"" Sep 10 18:37:19.137912 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.137739 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 18:37:19.137912 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.137760 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 18:37:19.137912 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.137539 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 18:37:19.137912 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.137797 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 18:37:19.137912 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.137830 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 18:37:19.140768 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.140752 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 18:37:19.143014 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.142995 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 18:37:19.146145 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.146122 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 18:37:19.212881 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.212849 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213013 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.212893 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213013 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.212926 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213013 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.212953 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fe3490c4-a7a9-4e07-9302-af7790f1f68b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213013 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.212990 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fe3490c4-a7a9-4e07-9302-af7790f1f68b-config-out\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213013 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.213006 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fe3490c4-a7a9-4e07-9302-af7790f1f68b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213263 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.213039 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqvlf\" (UniqueName: \"kubernetes.io/projected/fe3490c4-a7a9-4e07-9302-af7790f1f68b-kube-api-access-cqvlf\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213263 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.213079 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213263 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.213150 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213263 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.213183 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213263 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.213213 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213263 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.213243 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-web-config\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213534 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.213269 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213534 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.213301 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213534 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.213332 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213534 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.213390 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213534 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.213427 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213534 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.213452 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213534 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.213499 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-config\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.213828 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.213541 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.314368 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314332 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.314368 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314367 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.314583 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314385 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.314583 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314527 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.314583 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314568 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-web-config\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.314732 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314602 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.314732 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314634 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.314732 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314661 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.314732 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314699 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.314931 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314736 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.314931 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314766 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.314931 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314810 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-config\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.314931 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314830 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.314931 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314868 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.314931 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314902 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.315274 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314949 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.315274 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314977 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fe3490c4-a7a9-4e07-9302-af7790f1f68b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.315274 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.314991 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.315274 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.315006 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fe3490c4-a7a9-4e07-9302-af7790f1f68b-config-out\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.315274 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.315043 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fe3490c4-a7a9-4e07-9302-af7790f1f68b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.315274 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.315090 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqvlf\" (UniqueName: \"kubernetes.io/projected/fe3490c4-a7a9-4e07-9302-af7790f1f68b-kube-api-access-cqvlf\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.315719 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.315695 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fe3490c4-a7a9-4e07-9302-af7790f1f68b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.316339 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.316314 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.317599 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.317259 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-web-config\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.317599 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.317353 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fe3490c4-a7a9-4e07-9302-af7790f1f68b-config-out\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.317599 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.317554 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.317806 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.317700 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.317806 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.317780 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.318670 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.318080 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.318670 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.318348 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.318670 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.318503 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.318670 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.318610 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.318938 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.318793 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe3490c4-a7a9-4e07-9302-af7790f1f68b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.318994 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.318953 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-config\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.319600 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.319574 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.319780 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.319761 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.319893 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.319876 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fe3490c4-a7a9-4e07-9302-af7790f1f68b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.320580 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.320561 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.320766 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.320750 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fe3490c4-a7a9-4e07-9302-af7790f1f68b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.323266 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.323250 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqvlf\" (UniqueName: \"kubernetes.io/projected/fe3490c4-a7a9-4e07-9302-af7790f1f68b-kube-api-access-cqvlf\") pod \"prometheus-k8s-0\" (UID: \"fe3490c4-a7a9-4e07-9302-af7790f1f68b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.444028 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.444000 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:19.446845 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.446818 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4541aa2-c3d6-4af7-bd34-de202b1375b2" path="/var/lib/kubelet/pods/a4541aa2-c3d6-4af7-bd34-de202b1375b2/volumes" Sep 10 18:37:19.725837 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.725551 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 18:37:19.728086 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:37:19.728056 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe3490c4_a7a9_4e07_9302_af7790f1f68b.slice/crio-45df9ce524718fb6bee96d0abdae4e3780b1cf754d46317ee4c1024249210ce2 WatchSource:0}: Error finding container 45df9ce524718fb6bee96d0abdae4e3780b1cf754d46317ee4c1024249210ce2: Status 404 returned error can't find the container with id 45df9ce524718fb6bee96d0abdae4e3780b1cf754d46317ee4c1024249210ce2 Sep 10 18:37:19.777877 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:19.777854 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe3490c4-a7a9-4e07-9302-af7790f1f68b","Type":"ContainerStarted","Data":"45df9ce524718fb6bee96d0abdae4e3780b1cf754d46317ee4c1024249210ce2"} Sep 10 18:37:20.785521 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:20.785468 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" event={"ID":"73e86922-fc64-445d-a85f-0f0c4b849ac2","Type":"ContainerStarted","Data":"3d7d032b416f5b69de9f435d0b894bb328c3f787e57136e2eb0ba5efc02e1520"} Sep 10 18:37:20.785947 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:20.785580 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" event={"ID":"73e86922-fc64-445d-a85f-0f0c4b849ac2","Type":"ContainerStarted","Data":"f17b6a7d859756c8b74c5fda372e45f4fd67332d46ea93fa7d31097dcc5133cc"} Sep 10 18:37:20.785947 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:20.785607 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" event={"ID":"73e86922-fc64-445d-a85f-0f0c4b849ac2","Type":"ContainerStarted","Data":"237889120cbbc54ccd0eb7bc2f03eebc5b83f5ca30896b4a43c6338a9798193c"} Sep 10 18:37:20.787075 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:20.787052 2564 generic.go:356] "Generic (PLEG): container finished" podID="fe3490c4-a7a9-4e07-9302-af7790f1f68b" containerID="f7a136769d4897f18ef452c695c830ecbe53bb72209f60835f3264f9407f48a3" exitCode=0 Sep 10 18:37:20.787165 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:20.787094 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe3490c4-a7a9-4e07-9302-af7790f1f68b","Type":"ContainerDied","Data":"f7a136769d4897f18ef452c695c830ecbe53bb72209f60835f3264f9407f48a3"} Sep 10 18:37:20.805472 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:20.805426 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7f59bbb4cb-wvk9c" podStartSLOduration=1.290442448 podStartE2EDuration="3.805410275s" totalImagesPullingTime="2.514967827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:37:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:37:20.803104247 +0000 UTC m=+369.946407485" watchObservedRunningTime="2026-09-10 18:37:20.805410275 +0000 UTC m=+369.948713520" Sep 10 18:37:21.795638 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:21.795598 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe3490c4-a7a9-4e07-9302-af7790f1f68b","Type":"ContainerStarted","Data":"46261b5cf4243c1cb1d4c44733d0789cdea85999746abd9c0f20b1a44b740d60"} Sep 10 18:37:21.796320 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:21.795681 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe3490c4-a7a9-4e07-9302-af7790f1f68b","Type":"ContainerStarted","Data":"1eae6be7e859e34f9a45f975d7a1848f88e0a5a2b924dbf82820b3f7395986b9"} Sep 10 18:37:21.796320 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:21.795721 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe3490c4-a7a9-4e07-9302-af7790f1f68b","Type":"ContainerStarted","Data":"c6bae1e1635b11afc6c136c3cc22a0cd506850baa0c0e5b75d68f04c2bc08b67"} Sep 10 18:37:21.796320 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:21.795762 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe3490c4-a7a9-4e07-9302-af7790f1f68b","Type":"ContainerStarted","Data":"31bf134d2d3237c99d84cb8b4ee6c4bd9940825f58dc4f02c1aef2efe6b4e02f"} Sep 10 18:37:21.796320 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:21.795810 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe3490c4-a7a9-4e07-9302-af7790f1f68b","Type":"ContainerStarted","Data":"4082da12aca9eafb8c5582bb782264f587aa974a5123ca33ffadb1cecc16bc69"} Sep 10 18:37:21.796320 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:21.795846 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fe3490c4-a7a9-4e07-9302-af7790f1f68b","Type":"ContainerStarted","Data":"c70d6d0482337b4db124350de881f4342c455d10182323da1d43bf744e236b46"} Sep 10 18:37:21.820071 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:21.819998 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.7965656020000003 podStartE2EDuration="2.819983467s" totalImagesPullingTime="0s" totalInitContainerRuntime="23.417865ms" isStatefulPod=false podCreationTimestamp="2026-09-10 18:37:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:37:21.818751122 +0000 UTC m=+370.962054410" watchObservedRunningTime="2026-09-10 18:37:21.819983467 +0000 UTC m=+370.963286710" Sep 10 18:37:24.444965 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:24.444928 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:37:25.273477 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:25.273441 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs\") pod \"network-metrics-daemon-6dxjk\" (UID: \"99da651c-7b27-49ea-b49a-7f72caf21100\") " pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:37:25.275817 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:25.275794 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99da651c-7b27-49ea-b49a-7f72caf21100-metrics-certs\") pod \"network-metrics-daemon-6dxjk\" (UID: \"99da651c-7b27-49ea-b49a-7f72caf21100\") " pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:37:25.444285 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:25.444258 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-thvkz\"" Sep 10 18:37:25.453030 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:25.453006 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6dxjk" Sep 10 18:37:25.721310 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:25.721135 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6dxjk"] Sep 10 18:37:25.723874 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:37:25.723845 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99da651c_7b27_49ea_b49a_7f72caf21100.slice/crio-a82587d5cc18646455adf8a57140939c739a9f055ea202c13856010e18107283 WatchSource:0}: Error finding container a82587d5cc18646455adf8a57140939c739a9f055ea202c13856010e18107283: Status 404 returned error can't find the container with id a82587d5cc18646455adf8a57140939c739a9f055ea202c13856010e18107283 Sep 10 18:37:25.813223 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:25.813189 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6dxjk" event={"ID":"99da651c-7b27-49ea-b49a-7f72caf21100","Type":"ContainerStarted","Data":"a82587d5cc18646455adf8a57140939c739a9f055ea202c13856010e18107283"} Sep 10 18:37:27.825851 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:27.825802 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6dxjk" event={"ID":"99da651c-7b27-49ea-b49a-7f72caf21100","Type":"ContainerStarted","Data":"0351aed22011a5a89d9e000a7956a338bd0c1322b6ac699432d5e5d37310e496"} Sep 10 18:37:27.826332 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:27.825866 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6dxjk" event={"ID":"99da651c-7b27-49ea-b49a-7f72caf21100","Type":"ContainerStarted","Data":"a769ff8e602ce6f3fbfaab2b51ed76f66f165a4b4cc20835a2da987b0617ae20"} Sep 10 18:37:27.839930 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:37:27.839883 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6dxjk" podStartSLOduration=375.791465157 podStartE2EDuration="6m16.839865949s" totalImagesPullingTime="1.048400792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:37:27.839186698 +0000 UTC m=+376.982489942" watchObservedRunningTime="2026-09-10 18:37:27.839865949 +0000 UTC m=+376.983169215" Sep 10 18:38:19.447496 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:38:19.447447 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:38:19.460655 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:38:19.460629 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:38:20.051167 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:38:20.051138 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 18:40:33.298152 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:33.298115 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpvjh"] Sep 10 18:40:33.301293 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:33.301275 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpvjh" Sep 10 18:40:33.303353 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:33.303331 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-68lzz\"" Sep 10 18:40:33.303890 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:33.303873 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 10 18:40:33.303945 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:33.303912 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 10 18:40:33.307922 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:33.307894 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpvjh"] Sep 10 18:40:33.442974 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:33.442937 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2sh8\" (UniqueName: \"kubernetes.io/projected/4750f6be-a2e4-4a80-9347-d4afa70f2105-kube-api-access-s2sh8\") pod \"openshift-lws-operator-fd8ccff4c-cpvjh\" (UID: \"4750f6be-a2e4-4a80-9347-d4afa70f2105\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpvjh" Sep 10 18:40:33.442974 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:33.442982 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4750f6be-a2e4-4a80-9347-d4afa70f2105-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cpvjh\" (UID: \"4750f6be-a2e4-4a80-9347-d4afa70f2105\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpvjh" Sep 10 18:40:33.544108 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:33.544068 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2sh8\" (UniqueName: \"kubernetes.io/projected/4750f6be-a2e4-4a80-9347-d4afa70f2105-kube-api-access-s2sh8\") pod \"openshift-lws-operator-fd8ccff4c-cpvjh\" (UID: \"4750f6be-a2e4-4a80-9347-d4afa70f2105\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpvjh" Sep 10 18:40:33.544306 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:33.544123 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4750f6be-a2e4-4a80-9347-d4afa70f2105-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cpvjh\" (UID: \"4750f6be-a2e4-4a80-9347-d4afa70f2105\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpvjh" Sep 10 18:40:33.544560 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:33.544538 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4750f6be-a2e4-4a80-9347-d4afa70f2105-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cpvjh\" (UID: \"4750f6be-a2e4-4a80-9347-d4afa70f2105\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpvjh" Sep 10 18:40:33.551851 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:33.551804 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2sh8\" (UniqueName: \"kubernetes.io/projected/4750f6be-a2e4-4a80-9347-d4afa70f2105-kube-api-access-s2sh8\") pod \"openshift-lws-operator-fd8ccff4c-cpvjh\" (UID: \"4750f6be-a2e4-4a80-9347-d4afa70f2105\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpvjh" Sep 10 18:40:33.621456 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:33.621419 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpvjh" Sep 10 18:40:33.889709 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:33.889588 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpvjh"] Sep 10 18:40:33.893631 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:40:33.893603 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4750f6be_a2e4_4a80_9347_d4afa70f2105.slice/crio-72abbf55f520aaa7320476486753ab2af022ea1cbe029fe42c2a0bfa9e5eab89 WatchSource:0}: Error finding container 72abbf55f520aaa7320476486753ab2af022ea1cbe029fe42c2a0bfa9e5eab89: Status 404 returned error can't find the container with id 72abbf55f520aaa7320476486753ab2af022ea1cbe029fe42c2a0bfa9e5eab89 Sep 10 18:40:34.570728 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:34.570693 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpvjh" event={"ID":"4750f6be-a2e4-4a80-9347-d4afa70f2105","Type":"ContainerStarted","Data":"72abbf55f520aaa7320476486753ab2af022ea1cbe029fe42c2a0bfa9e5eab89"} Sep 10 18:40:39.595502 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:39.595444 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpvjh" event={"ID":"4750f6be-a2e4-4a80-9347-d4afa70f2105","Type":"ContainerStarted","Data":"fdc86c07a4574fdbc5a0a46888725e23706ca69616288d2afb2ade1442df6031"} Sep 10 18:40:39.611931 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:39.611876 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpvjh" podStartSLOduration=1.537924137 podStartE2EDuration="6.61184847s" totalImagesPullingTime="5.073924333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:40:39.609396199 +0000 UTC m=+568.752699454" watchObservedRunningTime="2026-09-10 18:40:39.61184847 +0000 UTC m=+568.755151734" Sep 10 18:40:48.382648 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.382610 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz"] Sep 10 18:40:48.386256 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.386234 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" Sep 10 18:40:48.388859 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.388833 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 10 18:40:48.388998 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.388866 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 10 18:40:48.388998 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.388870 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 10 18:40:48.388998 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.388913 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bf6pz\"" Sep 10 18:40:48.397307 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.397283 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz"] Sep 10 18:40:48.475051 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.475014 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fad3eec9-27b2-496e-8be5-29c592111646-cert\") pod \"lws-controller-manager-ddb978bf8-bzbnz\" (UID: \"fad3eec9-27b2-496e-8be5-29c592111646\") " pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" Sep 10 18:40:48.475262 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.475085 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb7wl\" (UniqueName: \"kubernetes.io/projected/fad3eec9-27b2-496e-8be5-29c592111646-kube-api-access-kb7wl\") pod \"lws-controller-manager-ddb978bf8-bzbnz\" (UID: \"fad3eec9-27b2-496e-8be5-29c592111646\") " pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" Sep 10 18:40:48.475262 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.475106 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fad3eec9-27b2-496e-8be5-29c592111646-metrics-cert\") pod \"lws-controller-manager-ddb978bf8-bzbnz\" (UID: \"fad3eec9-27b2-496e-8be5-29c592111646\") " pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" Sep 10 18:40:48.475262 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.475122 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fad3eec9-27b2-496e-8be5-29c592111646-manager-config\") pod \"lws-controller-manager-ddb978bf8-bzbnz\" (UID: \"fad3eec9-27b2-496e-8be5-29c592111646\") " pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" Sep 10 18:40:48.576530 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.576477 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fad3eec9-27b2-496e-8be5-29c592111646-cert\") pod \"lws-controller-manager-ddb978bf8-bzbnz\" (UID: \"fad3eec9-27b2-496e-8be5-29c592111646\") " pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" Sep 10 18:40:48.576694 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.576561 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb7wl\" (UniqueName: \"kubernetes.io/projected/fad3eec9-27b2-496e-8be5-29c592111646-kube-api-access-kb7wl\") pod \"lws-controller-manager-ddb978bf8-bzbnz\" (UID: \"fad3eec9-27b2-496e-8be5-29c592111646\") " pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" Sep 10 18:40:48.576694 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.576580 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fad3eec9-27b2-496e-8be5-29c592111646-metrics-cert\") pod \"lws-controller-manager-ddb978bf8-bzbnz\" (UID: \"fad3eec9-27b2-496e-8be5-29c592111646\") " pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" Sep 10 18:40:48.576694 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.576599 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fad3eec9-27b2-496e-8be5-29c592111646-manager-config\") pod \"lws-controller-manager-ddb978bf8-bzbnz\" (UID: \"fad3eec9-27b2-496e-8be5-29c592111646\") " pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" Sep 10 18:40:48.577248 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.577212 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fad3eec9-27b2-496e-8be5-29c592111646-manager-config\") pod \"lws-controller-manager-ddb978bf8-bzbnz\" (UID: \"fad3eec9-27b2-496e-8be5-29c592111646\") " pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" Sep 10 18:40:48.579045 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.579023 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fad3eec9-27b2-496e-8be5-29c592111646-cert\") pod \"lws-controller-manager-ddb978bf8-bzbnz\" (UID: \"fad3eec9-27b2-496e-8be5-29c592111646\") " pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" Sep 10 18:40:48.579126 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.579109 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fad3eec9-27b2-496e-8be5-29c592111646-metrics-cert\") pod \"lws-controller-manager-ddb978bf8-bzbnz\" (UID: \"fad3eec9-27b2-496e-8be5-29c592111646\") " pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" Sep 10 18:40:48.583852 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.583834 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb7wl\" (UniqueName: \"kubernetes.io/projected/fad3eec9-27b2-496e-8be5-29c592111646-kube-api-access-kb7wl\") pod \"lws-controller-manager-ddb978bf8-bzbnz\" (UID: \"fad3eec9-27b2-496e-8be5-29c592111646\") " pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" Sep 10 18:40:48.695919 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.695829 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" Sep 10 18:40:48.967992 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:48.966708 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz"] Sep 10 18:40:48.971209 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:40:48.971182 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfad3eec9_27b2_496e_8be5_29c592111646.slice/crio-bdc306a6328e8ff65551b61dbcfac714b196c80fa9d45443b1f24b425490a85f WatchSource:0}: Error finding container bdc306a6328e8ff65551b61dbcfac714b196c80fa9d45443b1f24b425490a85f: Status 404 returned error can't find the container with id bdc306a6328e8ff65551b61dbcfac714b196c80fa9d45443b1f24b425490a85f Sep 10 18:40:49.638887 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:49.638850 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" event={"ID":"fad3eec9-27b2-496e-8be5-29c592111646","Type":"ContainerStarted","Data":"bdc306a6328e8ff65551b61dbcfac714b196c80fa9d45443b1f24b425490a85f"} Sep 10 18:40:51.653366 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:51.653328 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" event={"ID":"fad3eec9-27b2-496e-8be5-29c592111646","Type":"ContainerStarted","Data":"aae21a710cc8b7280f4da227a78e27954daf2b3d989da01aef84cbab67e1ef5d"} Sep 10 18:40:51.653801 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:51.653436 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" Sep 10 18:40:51.673928 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:51.673875 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" podStartSLOduration=1.950983909 podStartE2EDuration="3.673858691s" totalImagesPullingTime="1.722874782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:40:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:40:51.672932935 +0000 UTC m=+580.816236178" watchObservedRunningTime="2026-09-10 18:40:51.673858691 +0000 UTC m=+580.817162019" Sep 10 18:40:54.567837 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.567804 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk"] Sep 10 18:40:54.574778 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.574743 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:54.577837 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.577809 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 10 18:40:54.577981 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.577835 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 10 18:40:54.577981 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.577837 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2hqf5\"" Sep 10 18:40:54.577981 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.577881 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 10 18:40:54.577981 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.577895 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 10 18:40:54.595531 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.595501 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk"] Sep 10 18:40:54.633242 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.633202 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4l5t\" (UniqueName: \"kubernetes.io/projected/0b6bccde-cc93-4199-9007-68fba5ed46cd-kube-api-access-v4l5t\") pod \"opendatahub-operator-controller-manager-bb8cfcdd6-28jxk\" (UID: \"0b6bccde-cc93-4199-9007-68fba5ed46cd\") " pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:54.633438 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.633279 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b6bccde-cc93-4199-9007-68fba5ed46cd-tmp\") pod \"opendatahub-operator-controller-manager-bb8cfcdd6-28jxk\" (UID: \"0b6bccde-cc93-4199-9007-68fba5ed46cd\") " pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:54.633438 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.633332 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b6bccde-cc93-4199-9007-68fba5ed46cd-webhook-cert\") pod \"opendatahub-operator-controller-manager-bb8cfcdd6-28jxk\" (UID: \"0b6bccde-cc93-4199-9007-68fba5ed46cd\") " pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:54.633438 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.633412 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0b6bccde-cc93-4199-9007-68fba5ed46cd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bb8cfcdd6-28jxk\" (UID: \"0b6bccde-cc93-4199-9007-68fba5ed46cd\") " pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:54.633609 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.633444 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0b6bccde-cc93-4199-9007-68fba5ed46cd-manifests\") pod \"opendatahub-operator-controller-manager-bb8cfcdd6-28jxk\" (UID: \"0b6bccde-cc93-4199-9007-68fba5ed46cd\") " pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:54.734446 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.734403 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b6bccde-cc93-4199-9007-68fba5ed46cd-tmp\") pod \"opendatahub-operator-controller-manager-bb8cfcdd6-28jxk\" (UID: \"0b6bccde-cc93-4199-9007-68fba5ed46cd\") " pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:54.734446 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.734448 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b6bccde-cc93-4199-9007-68fba5ed46cd-webhook-cert\") pod \"opendatahub-operator-controller-manager-bb8cfcdd6-28jxk\" (UID: \"0b6bccde-cc93-4199-9007-68fba5ed46cd\") " pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:54.734714 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.734520 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0b6bccde-cc93-4199-9007-68fba5ed46cd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bb8cfcdd6-28jxk\" (UID: \"0b6bccde-cc93-4199-9007-68fba5ed46cd\") " pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:54.734714 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.734541 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0b6bccde-cc93-4199-9007-68fba5ed46cd-manifests\") pod \"opendatahub-operator-controller-manager-bb8cfcdd6-28jxk\" (UID: \"0b6bccde-cc93-4199-9007-68fba5ed46cd\") " pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:54.734714 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.734565 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4l5t\" (UniqueName: \"kubernetes.io/projected/0b6bccde-cc93-4199-9007-68fba5ed46cd-kube-api-access-v4l5t\") pod \"opendatahub-operator-controller-manager-bb8cfcdd6-28jxk\" (UID: \"0b6bccde-cc93-4199-9007-68fba5ed46cd\") " pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:54.735021 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.734995 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0b6bccde-cc93-4199-9007-68fba5ed46cd-manifests\") pod \"opendatahub-operator-controller-manager-bb8cfcdd6-28jxk\" (UID: \"0b6bccde-cc93-4199-9007-68fba5ed46cd\") " pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:54.736756 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.736723 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b6bccde-cc93-4199-9007-68fba5ed46cd-tmp\") pod \"opendatahub-operator-controller-manager-bb8cfcdd6-28jxk\" (UID: \"0b6bccde-cc93-4199-9007-68fba5ed46cd\") " pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:54.737053 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.737035 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0b6bccde-cc93-4199-9007-68fba5ed46cd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bb8cfcdd6-28jxk\" (UID: \"0b6bccde-cc93-4199-9007-68fba5ed46cd\") " pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:54.737118 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.737047 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b6bccde-cc93-4199-9007-68fba5ed46cd-webhook-cert\") pod \"opendatahub-operator-controller-manager-bb8cfcdd6-28jxk\" (UID: \"0b6bccde-cc93-4199-9007-68fba5ed46cd\") " pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:54.741298 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.741276 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4l5t\" (UniqueName: \"kubernetes.io/projected/0b6bccde-cc93-4199-9007-68fba5ed46cd-kube-api-access-v4l5t\") pod \"opendatahub-operator-controller-manager-bb8cfcdd6-28jxk\" (UID: \"0b6bccde-cc93-4199-9007-68fba5ed46cd\") " pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:54.885115 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:54.885019 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:40:55.169267 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:55.169238 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk"] Sep 10 18:40:55.172288 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:40:55.172256 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b6bccde_cc93_4199_9007_68fba5ed46cd.slice/crio-d6bb0419f9cbdc2bbc384238ca011828aef60139633822e08b5cacdda3e2ef21 WatchSource:0}: Error finding container d6bb0419f9cbdc2bbc384238ca011828aef60139633822e08b5cacdda3e2ef21: Status 404 returned error can't find the container with id d6bb0419f9cbdc2bbc384238ca011828aef60139633822e08b5cacdda3e2ef21 Sep 10 18:40:55.672680 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:55.672642 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" event={"ID":"0b6bccde-cc93-4199-9007-68fba5ed46cd","Type":"ContainerStarted","Data":"d6bb0419f9cbdc2bbc384238ca011828aef60139633822e08b5cacdda3e2ef21"} Sep 10 18:40:58.689148 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:58.689107 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" event={"ID":"0b6bccde-cc93-4199-9007-68fba5ed46cd","Type":"ContainerStarted","Data":"a46b8dd9210ef61ad14cd8c16527fd55f010837749410b5ec4a17e4fe1197864"} Sep 10 18:40:59.695308 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:59.695227 2564 generic.go:356] "Generic (PLEG): container finished" podID="0b6bccde-cc93-4199-9007-68fba5ed46cd" containerID="a46b8dd9210ef61ad14cd8c16527fd55f010837749410b5ec4a17e4fe1197864" exitCode=0 Sep 10 18:40:59.695308 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:40:59.695293 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" event={"ID":"0b6bccde-cc93-4199-9007-68fba5ed46cd","Type":"ContainerDied","Data":"a46b8dd9210ef61ad14cd8c16527fd55f010837749410b5ec4a17e4fe1197864"} Sep 10 18:41:00.702422 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:00.702382 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" event={"ID":"0b6bccde-cc93-4199-9007-68fba5ed46cd","Type":"ContainerStarted","Data":"2283f429a21205f7f957fb7f9f4a054767656dfcd76d8c418a5c40cc93b1fd2a"} Sep 10 18:41:00.702927 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:00.702498 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:41:00.722187 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:00.722137 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" podStartSLOduration=2.327508843 podStartE2EDuration="6.722122208s" totalImagesPullingTime="3.311220101s" totalInitContainerRuntime="1.083393264s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:40:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:41:00.720057072 +0000 UTC m=+589.863360336" watchObservedRunningTime="2026-09-10 18:41:00.722122208 +0000 UTC m=+589.865425452" Sep 10 18:41:02.660204 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:02.660166 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-ddb978bf8-bzbnz" Sep 10 18:41:11.708349 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:11.708314 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-bb8cfcdd6-28jxk" Sep 10 18:41:14.016270 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.016232 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d28xz_3c15c29b-edff-4386-a50a-177b41560acd/console-operator/2.log" Sep 10 18:41:14.018206 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.017895 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d28xz_3c15c29b-edff-4386-a50a-177b41560acd/console-operator/2.log" Sep 10 18:41:14.024620 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.024600 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/ovn-acl-logging/0.log" Sep 10 18:41:14.026106 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.026089 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/ovn-acl-logging/0.log" Sep 10 18:41:14.143316 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.143284 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w"] Sep 10 18:41:14.146539 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.146524 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w" Sep 10 18:41:14.148583 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.148555 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 10 18:41:14.148705 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.148608 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 18:41:14.149136 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.149119 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vgtwx\"" Sep 10 18:41:14.149136 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.149132 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 10 18:41:14.149266 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.149148 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 18:41:14.159777 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.159755 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w"] Sep 10 18:41:14.308474 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.308379 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e43c3f93-a82d-4f05-a2e3-48ac25b6efae-tmp\") pod \"kube-auth-proxy-64dfbcccbf-mmw7w\" (UID: \"e43c3f93-a82d-4f05-a2e3-48ac25b6efae\") " pod="openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w" Sep 10 18:41:14.308474 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.308410 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5vlq\" (UniqueName: \"kubernetes.io/projected/e43c3f93-a82d-4f05-a2e3-48ac25b6efae-kube-api-access-h5vlq\") pod \"kube-auth-proxy-64dfbcccbf-mmw7w\" (UID: \"e43c3f93-a82d-4f05-a2e3-48ac25b6efae\") " pod="openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w" Sep 10 18:41:14.308474 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.308434 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e43c3f93-a82d-4f05-a2e3-48ac25b6efae-tls-certs\") pod \"kube-auth-proxy-64dfbcccbf-mmw7w\" (UID: \"e43c3f93-a82d-4f05-a2e3-48ac25b6efae\") " pod="openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w" Sep 10 18:41:14.409374 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.409317 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e43c3f93-a82d-4f05-a2e3-48ac25b6efae-tmp\") pod \"kube-auth-proxy-64dfbcccbf-mmw7w\" (UID: \"e43c3f93-a82d-4f05-a2e3-48ac25b6efae\") " pod="openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w" Sep 10 18:41:14.409374 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.409370 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5vlq\" (UniqueName: \"kubernetes.io/projected/e43c3f93-a82d-4f05-a2e3-48ac25b6efae-kube-api-access-h5vlq\") pod \"kube-auth-proxy-64dfbcccbf-mmw7w\" (UID: \"e43c3f93-a82d-4f05-a2e3-48ac25b6efae\") " pod="openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w" Sep 10 18:41:14.409629 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.409530 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e43c3f93-a82d-4f05-a2e3-48ac25b6efae-tls-certs\") pod \"kube-auth-proxy-64dfbcccbf-mmw7w\" (UID: \"e43c3f93-a82d-4f05-a2e3-48ac25b6efae\") " pod="openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w" Sep 10 18:41:14.411689 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.411662 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e43c3f93-a82d-4f05-a2e3-48ac25b6efae-tmp\") pod \"kube-auth-proxy-64dfbcccbf-mmw7w\" (UID: \"e43c3f93-a82d-4f05-a2e3-48ac25b6efae\") " pod="openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w" Sep 10 18:41:14.411890 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.411872 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e43c3f93-a82d-4f05-a2e3-48ac25b6efae-tls-certs\") pod \"kube-auth-proxy-64dfbcccbf-mmw7w\" (UID: \"e43c3f93-a82d-4f05-a2e3-48ac25b6efae\") " pod="openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w" Sep 10 18:41:14.417377 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.417356 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5vlq\" (UniqueName: \"kubernetes.io/projected/e43c3f93-a82d-4f05-a2e3-48ac25b6efae-kube-api-access-h5vlq\") pod \"kube-auth-proxy-64dfbcccbf-mmw7w\" (UID: \"e43c3f93-a82d-4f05-a2e3-48ac25b6efae\") " pod="openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w" Sep 10 18:41:14.456438 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.456403 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w" Sep 10 18:41:14.727177 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.727152 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w"] Sep 10 18:41:14.729825 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:41:14.729786 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode43c3f93_a82d_4f05_a2e3_48ac25b6efae.slice/crio-3f946e0087e14b309d2c31abd941db95c2e358497c298f955df830abcac63f4a WatchSource:0}: Error finding container 3f946e0087e14b309d2c31abd941db95c2e358497c298f955df830abcac63f4a: Status 404 returned error can't find the container with id 3f946e0087e14b309d2c31abd941db95c2e358497c298f955df830abcac63f4a Sep 10 18:41:14.765312 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:14.765285 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w" event={"ID":"e43c3f93-a82d-4f05-a2e3-48ac25b6efae","Type":"ContainerStarted","Data":"3f946e0087e14b309d2c31abd941db95c2e358497c298f955df830abcac63f4a"} Sep 10 18:41:17.783697 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:17.783647 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w" event={"ID":"e43c3f93-a82d-4f05-a2e3-48ac25b6efae","Type":"ContainerStarted","Data":"15086b9b2e087bc5c99680d79dae825b0bfee14c201b346636a3bee26c1b4014"} Sep 10 18:41:17.799569 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:41:17.799520 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-64dfbcccbf-mmw7w" podStartSLOduration=0.933991483 podStartE2EDuration="3.799502238s" totalImagesPullingTime="2.865510755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:41:17.798678922 +0000 UTC m=+606.941982167" watchObservedRunningTime="2026-09-10 18:41:17.799502238 +0000 UTC m=+606.942805475" Sep 10 18:43:01.654095 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:01.654058 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8jpqf"] Sep 10 18:43:01.657607 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:01.657584 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8jpqf" Sep 10 18:43:01.659781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:01.659757 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 10 18:43:01.659903 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:01.659793 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 10 18:43:01.659903 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:01.659834 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-n4z5k\"" Sep 10 18:43:01.670020 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:01.669996 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8jpqf"] Sep 10 18:43:01.819877 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:01.819808 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8b49aaa-7628-48c6-823d-1122be9ca71e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8jpqf\" (UID: \"e8b49aaa-7628-48c6-823d-1122be9ca71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8jpqf" Sep 10 18:43:01.820041 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:01.819899 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmt7m\" (UniqueName: \"kubernetes.io/projected/e8b49aaa-7628-48c6-823d-1122be9ca71e-kube-api-access-wmt7m\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8jpqf\" (UID: \"e8b49aaa-7628-48c6-823d-1122be9ca71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8jpqf" Sep 10 18:43:01.920411 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:01.920317 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8b49aaa-7628-48c6-823d-1122be9ca71e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8jpqf\" (UID: \"e8b49aaa-7628-48c6-823d-1122be9ca71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8jpqf" Sep 10 18:43:01.920411 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:01.920373 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmt7m\" (UniqueName: \"kubernetes.io/projected/e8b49aaa-7628-48c6-823d-1122be9ca71e-kube-api-access-wmt7m\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8jpqf\" (UID: \"e8b49aaa-7628-48c6-823d-1122be9ca71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8jpqf" Sep 10 18:43:01.920746 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:01.920723 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8b49aaa-7628-48c6-823d-1122be9ca71e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8jpqf\" (UID: \"e8b49aaa-7628-48c6-823d-1122be9ca71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8jpqf" Sep 10 18:43:01.931221 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:01.931195 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmt7m\" (UniqueName: \"kubernetes.io/projected/e8b49aaa-7628-48c6-823d-1122be9ca71e-kube-api-access-wmt7m\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8jpqf\" (UID: \"e8b49aaa-7628-48c6-823d-1122be9ca71e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8jpqf" Sep 10 18:43:01.968184 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:01.968153 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8jpqf" Sep 10 18:43:02.251722 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:02.251698 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8jpqf"] Sep 10 18:43:02.254886 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:43:02.254858 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8b49aaa_7628_48c6_823d_1122be9ca71e.slice/crio-8d3e720d0183c454c2903ae39ae0583b0d895f379b7b8bb530303e47521da39a WatchSource:0}: Error finding container 8d3e720d0183c454c2903ae39ae0583b0d895f379b7b8bb530303e47521da39a: Status 404 returned error can't find the container with id 8d3e720d0183c454c2903ae39ae0583b0d895f379b7b8bb530303e47521da39a Sep 10 18:43:02.258086 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:02.258062 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 18:43:02.267774 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:02.267745 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8jpqf" event={"ID":"e8b49aaa-7628-48c6-823d-1122be9ca71e","Type":"ContainerStarted","Data":"8d3e720d0183c454c2903ae39ae0583b0d895f379b7b8bb530303e47521da39a"} Sep 10 18:43:07.298265 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:07.298228 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8jpqf" event={"ID":"e8b49aaa-7628-48c6-823d-1122be9ca71e","Type":"ContainerStarted","Data":"2f5cf72c9f4bb851b1985c8e6e93eaa0685bbbdfbb9ba718acc54735a2e3290f"} Sep 10 18:43:07.298755 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:07.298340 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8jpqf" Sep 10 18:43:07.319273 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:07.319220 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8jpqf" podStartSLOduration=2.306342693 podStartE2EDuration="6.319205813s" totalImagesPullingTime="4.01286312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:43:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:43:07.316676873 +0000 UTC m=+716.459980157" watchObservedRunningTime="2026-09-10 18:43:07.319205813 +0000 UTC m=+716.462509058" Sep 10 18:43:18.306082 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:18.306050 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8jpqf" Sep 10 18:43:40.201637 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.201601 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ckfpc"] Sep 10 18:43:40.207730 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.207698 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-ckfpc" Sep 10 18:43:40.209823 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.209791 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 10 18:43:40.210397 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.210370 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4zxlc\"" Sep 10 18:43:40.212520 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.212499 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ckfpc"] Sep 10 18:43:40.235170 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.235147 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ckfpc"] Sep 10 18:43:40.243868 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.243847 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9hxc\" (UniqueName: \"kubernetes.io/projected/bf301dad-e02f-427c-bf5f-b927271f1b35-kube-api-access-g9hxc\") pod \"limitador-limitador-84994f7ffc-ckfpc\" (UID: \"bf301dad-e02f-427c-bf5f-b927271f1b35\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ckfpc" Sep 10 18:43:40.243980 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.243958 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bf301dad-e02f-427c-bf5f-b927271f1b35-config-file\") pod \"limitador-limitador-84994f7ffc-ckfpc\" (UID: \"bf301dad-e02f-427c-bf5f-b927271f1b35\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ckfpc" Sep 10 18:43:40.345343 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.345303 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9hxc\" (UniqueName: \"kubernetes.io/projected/bf301dad-e02f-427c-bf5f-b927271f1b35-kube-api-access-g9hxc\") pod \"limitador-limitador-84994f7ffc-ckfpc\" (UID: \"bf301dad-e02f-427c-bf5f-b927271f1b35\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ckfpc" Sep 10 18:43:40.345554 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.345394 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bf301dad-e02f-427c-bf5f-b927271f1b35-config-file\") pod \"limitador-limitador-84994f7ffc-ckfpc\" (UID: \"bf301dad-e02f-427c-bf5f-b927271f1b35\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ckfpc" Sep 10 18:43:40.346084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.346059 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bf301dad-e02f-427c-bf5f-b927271f1b35-config-file\") pod \"limitador-limitador-84994f7ffc-ckfpc\" (UID: \"bf301dad-e02f-427c-bf5f-b927271f1b35\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ckfpc" Sep 10 18:43:40.352892 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.352870 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9hxc\" (UniqueName: \"kubernetes.io/projected/bf301dad-e02f-427c-bf5f-b927271f1b35-kube-api-access-g9hxc\") pod \"limitador-limitador-84994f7ffc-ckfpc\" (UID: \"bf301dad-e02f-427c-bf5f-b927271f1b35\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ckfpc" Sep 10 18:43:40.520258 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.520212 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-ckfpc" Sep 10 18:43:40.779700 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.779630 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-kfd94"] Sep 10 18:43:40.782944 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.782926 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-kfd94" Sep 10 18:43:40.785898 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.785877 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-v49zl\"" Sep 10 18:43:40.801171 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.801149 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kfd94"] Sep 10 18:43:40.804263 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:43:40.804237 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf301dad_e02f_427c_bf5f_b927271f1b35.slice/crio-1622880a506cf965d88b78c6d5969c5c1fe48b7bed8c9c84911be82004e749c2 WatchSource:0}: Error finding container 1622880a506cf965d88b78c6d5969c5c1fe48b7bed8c9c84911be82004e749c2: Status 404 returned error can't find the container with id 1622880a506cf965d88b78c6d5969c5c1fe48b7bed8c9c84911be82004e749c2 Sep 10 18:43:40.806001 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.805962 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ckfpc"] Sep 10 18:43:40.850277 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.850249 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwn59\" (UniqueName: \"kubernetes.io/projected/5d943869-e08a-457f-b636-24bc68e17dca-kube-api-access-rwn59\") pod \"authorino-59c974bd48-kfd94\" (UID: \"5d943869-e08a-457f-b636-24bc68e17dca\") " pod="kuadrant-system/authorino-59c974bd48-kfd94" Sep 10 18:43:40.951638 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.951604 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwn59\" (UniqueName: \"kubernetes.io/projected/5d943869-e08a-457f-b636-24bc68e17dca-kube-api-access-rwn59\") pod \"authorino-59c974bd48-kfd94\" (UID: \"5d943869-e08a-457f-b636-24bc68e17dca\") " pod="kuadrant-system/authorino-59c974bd48-kfd94" Sep 10 18:43:40.958674 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:40.958645 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwn59\" (UniqueName: \"kubernetes.io/projected/5d943869-e08a-457f-b636-24bc68e17dca-kube-api-access-rwn59\") pod \"authorino-59c974bd48-kfd94\" (UID: \"5d943869-e08a-457f-b636-24bc68e17dca\") " pod="kuadrant-system/authorino-59c974bd48-kfd94" Sep 10 18:43:41.092579 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:41.092468 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-kfd94" Sep 10 18:43:41.360309 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:41.360267 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kfd94"] Sep 10 18:43:41.363200 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:43:41.363174 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d943869_e08a_457f_b636_24bc68e17dca.slice/crio-df7dc77ab3b19a4b1b7b0e0c05d4d0b09d0f0adb08b51cad97d362b9423590d5 WatchSource:0}: Error finding container df7dc77ab3b19a4b1b7b0e0c05d4d0b09d0f0adb08b51cad97d362b9423590d5: Status 404 returned error can't find the container with id df7dc77ab3b19a4b1b7b0e0c05d4d0b09d0f0adb08b51cad97d362b9423590d5 Sep 10 18:43:41.458876 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:41.458845 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-kfd94" event={"ID":"5d943869-e08a-457f-b636-24bc68e17dca","Type":"ContainerStarted","Data":"df7dc77ab3b19a4b1b7b0e0c05d4d0b09d0f0adb08b51cad97d362b9423590d5"} Sep 10 18:43:41.460026 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:41.459999 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-ckfpc" event={"ID":"bf301dad-e02f-427c-bf5f-b927271f1b35","Type":"ContainerStarted","Data":"1622880a506cf965d88b78c6d5969c5c1fe48b7bed8c9c84911be82004e749c2"} Sep 10 18:43:42.468624 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:42.468586 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-ckfpc" event={"ID":"bf301dad-e02f-427c-bf5f-b927271f1b35","Type":"ContainerStarted","Data":"be05f3627616e0816a6ed42e3365fe14878934c6eef0b69099b35ccdaeb90934"} Sep 10 18:43:42.469076 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:42.468873 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-ckfpc" Sep 10 18:43:42.485254 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:42.485172 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-ckfpc" podStartSLOduration=1.328282139 podStartE2EDuration="2.485151958s" totalImagesPullingTime="1.156869819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:43:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:43:42.481808567 +0000 UTC m=+751.625111811" watchObservedRunningTime="2026-09-10 18:43:42.485151958 +0000 UTC m=+751.628455204" Sep 10 18:43:43.474877 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:43.474839 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-kfd94" event={"ID":"5d943869-e08a-457f-b636-24bc68e17dca","Type":"ContainerStarted","Data":"4d9ab4f5fd937f026c02923b007b355c9d7e9dd5839e3327dad32549da63b4ba"} Sep 10 18:43:43.488619 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:43.488569 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-kfd94" podStartSLOduration=1.759594742 podStartE2EDuration="3.488551739s" totalImagesPullingTime="1.728956997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:43:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:43:43.486877706 +0000 UTC m=+752.630180952" watchObservedRunningTime="2026-09-10 18:43:43.488551739 +0000 UTC m=+752.631854983" Sep 10 18:43:53.476179 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:43:53.476149 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-ckfpc" Sep 10 18:44:09.137603 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:09.137569 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kfd94"] Sep 10 18:44:09.138008 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:09.137743 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-kfd94" podUID="5d943869-e08a-457f-b636-24bc68e17dca" containerName="authorino" containerID="cri-o://4d9ab4f5fd937f026c02923b007b355c9d7e9dd5839e3327dad32549da63b4ba" gracePeriod=30 Sep 10 18:44:09.374526 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:09.374472 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-kfd94" Sep 10 18:44:09.494901 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:09.494869 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwn59\" (UniqueName: \"kubernetes.io/projected/5d943869-e08a-457f-b636-24bc68e17dca-kube-api-access-rwn59\") pod \"5d943869-e08a-457f-b636-24bc68e17dca\" (UID: \"5d943869-e08a-457f-b636-24bc68e17dca\") " Sep 10 18:44:09.496919 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:09.496882 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d943869-e08a-457f-b636-24bc68e17dca-kube-api-access-rwn59" (OuterVolumeSpecName: "kube-api-access-rwn59") pod "5d943869-e08a-457f-b636-24bc68e17dca" (UID: "5d943869-e08a-457f-b636-24bc68e17dca"). InnerVolumeSpecName "kube-api-access-rwn59". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 18:44:09.595636 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:09.595611 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwn59\" (UniqueName: \"kubernetes.io/projected/5d943869-e08a-457f-b636-24bc68e17dca-kube-api-access-rwn59\") on node \"ip-10-0-138-87.ec2.internal\" DevicePath \"\"" Sep 10 18:44:09.596651 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:09.596627 2564 generic.go:356] "Generic (PLEG): container finished" podID="5d943869-e08a-457f-b636-24bc68e17dca" containerID="4d9ab4f5fd937f026c02923b007b355c9d7e9dd5839e3327dad32549da63b4ba" exitCode=0 Sep 10 18:44:09.596768 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:09.596681 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-kfd94" Sep 10 18:44:09.596768 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:09.596690 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-kfd94" event={"ID":"5d943869-e08a-457f-b636-24bc68e17dca","Type":"ContainerDied","Data":"4d9ab4f5fd937f026c02923b007b355c9d7e9dd5839e3327dad32549da63b4ba"} Sep 10 18:44:09.596768 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:09.596722 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-kfd94" event={"ID":"5d943869-e08a-457f-b636-24bc68e17dca","Type":"ContainerDied","Data":"df7dc77ab3b19a4b1b7b0e0c05d4d0b09d0f0adb08b51cad97d362b9423590d5"} Sep 10 18:44:09.596768 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:09.596743 2564 scope.go:117] "RemoveContainer" containerID="4d9ab4f5fd937f026c02923b007b355c9d7e9dd5839e3327dad32549da63b4ba" Sep 10 18:44:09.605656 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:09.605633 2564 scope.go:117] "RemoveContainer" containerID="4d9ab4f5fd937f026c02923b007b355c9d7e9dd5839e3327dad32549da63b4ba" Sep 10 18:44:09.605920 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:44:09.605896 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d9ab4f5fd937f026c02923b007b355c9d7e9dd5839e3327dad32549da63b4ba\": container with ID starting with 4d9ab4f5fd937f026c02923b007b355c9d7e9dd5839e3327dad32549da63b4ba not found: ID does not exist" containerID="4d9ab4f5fd937f026c02923b007b355c9d7e9dd5839e3327dad32549da63b4ba" Sep 10 18:44:09.606004 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:09.605926 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d9ab4f5fd937f026c02923b007b355c9d7e9dd5839e3327dad32549da63b4ba"} err="failed to get container status \"4d9ab4f5fd937f026c02923b007b355c9d7e9dd5839e3327dad32549da63b4ba\": rpc error: code = NotFound desc = could not find container \"4d9ab4f5fd937f026c02923b007b355c9d7e9dd5839e3327dad32549da63b4ba\": container with ID starting with 4d9ab4f5fd937f026c02923b007b355c9d7e9dd5839e3327dad32549da63b4ba not found: ID does not exist" Sep 10 18:44:09.618947 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:09.618914 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kfd94"] Sep 10 18:44:09.627401 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:09.627380 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kfd94"] Sep 10 18:44:11.447990 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:11.447954 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d943869-e08a-457f-b636-24bc68e17dca" path="/var/lib/kubelet/pods/5d943869-e08a-457f-b636-24bc68e17dca/volumes" Sep 10 18:44:53.586717 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:44:53.586635 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ckfpc"] Sep 10 18:45:16.670051 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:16.670008 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ckfpc"] Sep 10 18:45:33.570700 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:33.570664 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ckfpc"] Sep 10 18:45:47.266391 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:47.266342 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ckfpc"] Sep 10 18:45:55.864249 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:55.864216 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp"] Sep 10 18:45:55.864626 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:55.864611 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d943869-e08a-457f-b636-24bc68e17dca" containerName="authorino" Sep 10 18:45:55.864669 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:55.864630 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d943869-e08a-457f-b636-24bc68e17dca" containerName="authorino" Sep 10 18:45:55.864703 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:55.864690 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d943869-e08a-457f-b636-24bc68e17dca" containerName="authorino" Sep 10 18:45:55.867657 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:55.867637 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:55.869724 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:55.869704 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 10 18:45:55.870668 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:55.870640 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 10 18:45:55.870668 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:55.870655 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9zgnh\"" Sep 10 18:45:55.870907 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:55.870893 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 10 18:45:55.877321 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:55.877300 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp"] Sep 10 18:45:56.012621 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.012579 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.012621 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.012621 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.012992 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.012686 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.012992 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.012738 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b79kh\" (UniqueName: \"kubernetes.io/projected/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-kube-api-access-b79kh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.012992 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.012764 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.012992 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.012807 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.114026 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.113991 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.114026 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.114026 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.114301 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.114053 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.114301 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.114090 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b79kh\" (UniqueName: \"kubernetes.io/projected/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-kube-api-access-b79kh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.114301 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.114119 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.114301 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.114148 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.114538 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.114512 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.114623 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.114597 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.114664 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.114609 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.116646 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.116615 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.116911 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.116891 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.121412 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.121389 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b79kh\" (UniqueName: \"kubernetes.io/projected/8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7-kube-api-access-b79kh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-fv4bp\" (UID: \"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.181199 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.181154 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:45:56.452267 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:56.452244 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp"] Sep 10 18:45:56.454984 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:45:56.454949 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f9db1a3_5d09_4bc4_914f_8f5c1f3b62a7.slice/crio-97989c4bdd93e4a97a71740216873a00aea417e348212c68d5f5b0079500e1b4 WatchSource:0}: Error finding container 97989c4bdd93e4a97a71740216873a00aea417e348212c68d5f5b0079500e1b4: Status 404 returned error can't find the container with id 97989c4bdd93e4a97a71740216873a00aea417e348212c68d5f5b0079500e1b4 Sep 10 18:45:57.071441 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:57.071400 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ckfpc"] Sep 10 18:45:57.115659 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:57.115618 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" event={"ID":"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7","Type":"ContainerStarted","Data":"97989c4bdd93e4a97a71740216873a00aea417e348212c68d5f5b0079500e1b4"} Sep 10 18:45:58.122398 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:58.122355 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" event={"ID":"8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7","Type":"ContainerStarted","Data":"4e5c278933a0f8ec66fd5e00f84848a5687877cfd93d81f8c9ec1048b5261bbe"} Sep 10 18:45:58.138640 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:45:58.138591 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" podStartSLOduration=2.05075104 podStartE2EDuration="3.13857684s" totalImagesPullingTime="1.0878258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:45:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:45:58.13647816 +0000 UTC m=+887.279781404" watchObservedRunningTime="2026-09-10 18:45:58.13857684 +0000 UTC m=+887.281880083" Sep 10 18:46:05.967370 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:05.967278 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ckfpc"] Sep 10 18:46:06.181752 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:06.181717 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:46:06.181927 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:06.181786 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:46:06.194083 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:06.194053 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:46:07.176886 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:07.176850 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-fv4bp" Sep 10 18:46:14.051173 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:14.051053 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d28xz_3c15c29b-edff-4386-a50a-177b41560acd/console-operator/2.log" Sep 10 18:46:14.070213 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:14.051759 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d28xz_3c15c29b-edff-4386-a50a-177b41560acd/console-operator/2.log" Sep 10 18:46:14.070213 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:14.063403 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/ovn-acl-logging/0.log" Sep 10 18:46:14.070213 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:14.064312 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/ovn-acl-logging/0.log" Sep 10 18:46:17.072057 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:17.072021 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ckfpc"] Sep 10 18:46:40.963079 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:40.963034 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb"] Sep 10 18:46:40.969590 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:40.968408 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:40.974768 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:40.974741 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Sep 10 18:46:40.982855 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:40.982812 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb"] Sep 10 18:46:41.110862 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.110818 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.110862 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.110860 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg75w\" (UniqueName: \"kubernetes.io/projected/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-kube-api-access-tg75w\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.111066 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.110888 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.111066 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.110993 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.111066 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.111041 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.111164 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.111105 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.212413 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.212375 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.212413 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.212414 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg75w\" (UniqueName: \"kubernetes.io/projected/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-kube-api-access-tg75w\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.212647 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.212549 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.212647 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.212632 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.212747 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.212690 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.212798 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.212752 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.212875 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.212853 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-home\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.213056 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.213033 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.213185 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.213086 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.215009 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.214992 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-dshm\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.215382 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.215360 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.219923 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.219900 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg75w\" (UniqueName: \"kubernetes.io/projected/b95e9aa7-c088-4327-b3c2-f659cb7f34ae-kube-api-access-tg75w\") pod \"premium-simulated-simulated-premium-kserve-5758cf6949-fvscb\" (UID: \"b95e9aa7-c088-4327-b3c2-f659cb7f34ae\") " pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.284228 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.284186 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:41.557181 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:41.557153 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb"] Sep 10 18:46:41.559803 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:46:41.559773 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb95e9aa7_c088_4327_b3c2_f659cb7f34ae.slice/crio-368c9606b5986af2652e41de9690b8b73e458aa7a0bd260122717764c52acaaa WatchSource:0}: Error finding container 368c9606b5986af2652e41de9690b8b73e458aa7a0bd260122717764c52acaaa: Status 404 returned error can't find the container with id 368c9606b5986af2652e41de9690b8b73e458aa7a0bd260122717764c52acaaa Sep 10 18:46:42.341610 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:42.341572 2564 generic.go:356] "Generic (PLEG): container finished" podID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" containerID="a63ff17cfac11d621df900d3bbda9e9360fd64c31a555817349f903fbbf133af" exitCode=0 Sep 10 18:46:42.342078 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:42.341657 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" event={"ID":"b95e9aa7-c088-4327-b3c2-f659cb7f34ae","Type":"ContainerDied","Data":"a63ff17cfac11d621df900d3bbda9e9360fd64c31a555817349f903fbbf133af"} Sep 10 18:46:42.342078 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:42.341704 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" event={"ID":"b95e9aa7-c088-4327-b3c2-f659cb7f34ae","Type":"ContainerStarted","Data":"368c9606b5986af2652e41de9690b8b73e458aa7a0bd260122717764c52acaaa"} Sep 10 18:46:42.342078 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:42.341962 2564 scope.go:117] "RemoveContainer" containerID="a63ff17cfac11d621df900d3bbda9e9360fd64c31a555817349f903fbbf133af" Sep 10 18:46:43.268616 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:43.268577 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ckfpc"] Sep 10 18:46:43.348873 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:43.348837 2564 generic.go:356] "Generic (PLEG): container finished" podID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" containerID="13b19bc9a1179de8dc16cfe69e6df04d175ce7532261997166525ea814af4161" exitCode=0 Sep 10 18:46:43.349311 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:43.348896 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" event={"ID":"b95e9aa7-c088-4327-b3c2-f659cb7f34ae","Type":"ContainerDied","Data":"13b19bc9a1179de8dc16cfe69e6df04d175ce7532261997166525ea814af4161"} Sep 10 18:46:43.349311 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:43.348935 2564 scope.go:117] "RemoveContainer" containerID="a63ff17cfac11d621df900d3bbda9e9360fd64c31a555817349f903fbbf133af" Sep 10 18:46:43.349311 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:43.349159 2564 scope.go:117] "RemoveContainer" containerID="13b19bc9a1179de8dc16cfe69e6df04d175ce7532261997166525ea814af4161" Sep 10 18:46:43.349470 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:46:43.349358 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:46:44.355950 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:44.355924 2564 scope.go:117] "RemoveContainer" containerID="13b19bc9a1179de8dc16cfe69e6df04d175ce7532261997166525ea814af4161" Sep 10 18:46:44.356328 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:46:44.356100 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:46:51.285146 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:51.285095 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:51.285146 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:51.285159 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:51.285692 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:51.285171 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:51.285692 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:51.285180 2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:46:51.285692 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:46:51.285574 2564 scope.go:117] "RemoveContainer" containerID="13b19bc9a1179de8dc16cfe69e6df04d175ce7532261997166525ea814af4161" Sep 10 18:46:51.285798 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:46:51.285727 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:47:05.447589 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:05.447557 2564 scope.go:117] "RemoveContainer" containerID="13b19bc9a1179de8dc16cfe69e6df04d175ce7532261997166525ea814af4161" Sep 10 18:47:06.458318 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:06.458276 2564 generic.go:356] "Generic (PLEG): container finished" podID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" containerID="bf0355feb4dc46d667225551429c3a77044ea103ca5e36624c4ec2d66c96111d" exitCode=0 Sep 10 18:47:06.458761 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:06.458348 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" event={"ID":"b95e9aa7-c088-4327-b3c2-f659cb7f34ae","Type":"ContainerDied","Data":"bf0355feb4dc46d667225551429c3a77044ea103ca5e36624c4ec2d66c96111d"} Sep 10 18:47:06.458761 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:06.458397 2564 scope.go:117] "RemoveContainer" containerID="13b19bc9a1179de8dc16cfe69e6df04d175ce7532261997166525ea814af4161" Sep 10 18:47:06.458862 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:06.458772 2564 scope.go:117] "RemoveContainer" containerID="bf0355feb4dc46d667225551429c3a77044ea103ca5e36624c4ec2d66c96111d" Sep 10 18:47:06.458962 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:47:06.458940 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:47:11.284868 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:11.284832 2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:47:11.284868 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:11.284878 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:47:11.285323 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:11.284894 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:47:11.285323 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:11.284905 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:47:11.285323 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:11.285229 2564 scope.go:117] "RemoveContainer" containerID="bf0355feb4dc46d667225551429c3a77044ea103ca5e36624c4ec2d66c96111d" Sep 10 18:47:11.285421 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:47:11.285372 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:47:26.442513 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:26.442444 2564 scope.go:117] "RemoveContainer" containerID="bf0355feb4dc46d667225551429c3a77044ea103ca5e36624c4ec2d66c96111d" Sep 10 18:47:27.564667 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:27.564629 2564 generic.go:356] "Generic (PLEG): container finished" podID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" containerID="4560811fce1c5f20eb25ed5c57dc61e5a8d07551556292ce35e50692c145b770" exitCode=0 Sep 10 18:47:27.565080 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:27.564705 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" event={"ID":"b95e9aa7-c088-4327-b3c2-f659cb7f34ae","Type":"ContainerDied","Data":"4560811fce1c5f20eb25ed5c57dc61e5a8d07551556292ce35e50692c145b770"} Sep 10 18:47:27.565080 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:27.564756 2564 scope.go:117] "RemoveContainer" containerID="bf0355feb4dc46d667225551429c3a77044ea103ca5e36624c4ec2d66c96111d" Sep 10 18:47:27.565185 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:27.565150 2564 scope.go:117] "RemoveContainer" containerID="4560811fce1c5f20eb25ed5c57dc61e5a8d07551556292ce35e50692c145b770" Sep 10 18:47:27.565359 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:47:27.565336 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:47:31.284858 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:31.284821 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:47:31.285242 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:31.284875 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:47:31.285242 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:31.284886 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:47:31.285242 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:31.284894 2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:47:31.285242 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:31.285229 2564 scope.go:117] "RemoveContainer" containerID="4560811fce1c5f20eb25ed5c57dc61e5a8d07551556292ce35e50692c145b770" Sep 10 18:47:31.285402 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:47:31.285384 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:47:44.441573 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:44.441496 2564 scope.go:117] "RemoveContainer" containerID="4560811fce1c5f20eb25ed5c57dc61e5a8d07551556292ce35e50692c145b770" Sep 10 18:47:44.442008 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:47:44.441686 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:47:56.442336 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:47:56.442304 2564 scope.go:117] "RemoveContainer" containerID="4560811fce1c5f20eb25ed5c57dc61e5a8d07551556292ce35e50692c145b770" Sep 10 18:47:56.442736 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:47:56.442467 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:48:09.442406 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:48:09.442367 2564 scope.go:117] "RemoveContainer" containerID="4560811fce1c5f20eb25ed5c57dc61e5a8d07551556292ce35e50692c145b770" Sep 10 18:48:09.442891 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:48:09.442793 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 18:48:09.772028 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:48:09.772003 2564 generic.go:356] "Generic (PLEG): container finished" podID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" containerID="defc2a5ec8f69e23d03e79bfae067a54fe736384f4b3bcf0ad735f5d7f3488d2" exitCode=0 Sep 10 18:48:09.772120 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:48:09.772054 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" event={"ID":"b95e9aa7-c088-4327-b3c2-f659cb7f34ae","Type":"ContainerDied","Data":"defc2a5ec8f69e23d03e79bfae067a54fe736384f4b3bcf0ad735f5d7f3488d2"} Sep 10 18:48:09.772120 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:48:09.772106 2564 scope.go:117] "RemoveContainer" containerID="4560811fce1c5f20eb25ed5c57dc61e5a8d07551556292ce35e50692c145b770" Sep 10 18:48:09.772454 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:48:09.772433 2564 scope.go:117] "RemoveContainer" containerID="defc2a5ec8f69e23d03e79bfae067a54fe736384f4b3bcf0ad735f5d7f3488d2" Sep 10 18:48:09.772653 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:48:09.772631 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:48:11.284336 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:48:11.284300 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:48:11.284336 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:48:11.284343 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:48:11.284794 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:48:11.284355 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:48:11.284794 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:48:11.284363 2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:48:11.284794 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:48:11.284698 2564 scope.go:117] "RemoveContainer" containerID="defc2a5ec8f69e23d03e79bfae067a54fe736384f4b3bcf0ad735f5d7f3488d2" Sep 10 18:48:11.284887 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:48:11.284849 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:48:24.442185 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:48:24.442154 2564 scope.go:117] "RemoveContainer" containerID="defc2a5ec8f69e23d03e79bfae067a54fe736384f4b3bcf0ad735f5d7f3488d2" Sep 10 18:48:24.442659 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:48:24.442328 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:48:39.442186 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:48:39.442148 2564 scope.go:117] "RemoveContainer" containerID="defc2a5ec8f69e23d03e79bfae067a54fe736384f4b3bcf0ad735f5d7f3488d2" Sep 10 18:48:39.442693 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:48:39.442356 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:48:51.444193 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:48:51.444154 2564 scope.go:117] "RemoveContainer" containerID="defc2a5ec8f69e23d03e79bfae067a54fe736384f4b3bcf0ad735f5d7f3488d2" Sep 10 18:48:51.444782 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:48:51.444380 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:49:02.442302 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:49:02.442267 2564 scope.go:117] "RemoveContainer" containerID="defc2a5ec8f69e23d03e79bfae067a54fe736384f4b3bcf0ad735f5d7f3488d2" Sep 10 18:49:02.442755 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:49:02.442435 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:49:13.442349 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:49:13.442266 2564 scope.go:117] "RemoveContainer" containerID="defc2a5ec8f69e23d03e79bfae067a54fe736384f4b3bcf0ad735f5d7f3488d2" Sep 10 18:49:13.442752 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:49:13.442424 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:49:26.441872 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:49:26.441835 2564 scope.go:117] "RemoveContainer" containerID="defc2a5ec8f69e23d03e79bfae067a54fe736384f4b3bcf0ad735f5d7f3488d2" Sep 10 18:49:26.442376 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:49:26.442016 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:49:38.441601 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:49:38.441568 2564 scope.go:117] "RemoveContainer" containerID="defc2a5ec8f69e23d03e79bfae067a54fe736384f4b3bcf0ad735f5d7f3488d2" Sep 10 18:49:39.202525 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:49:39.202425 2564 generic.go:356] "Generic (PLEG): container finished" podID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" containerID="a2e1d2c39baaae733bfe2dd6de7677d8a29de21f1ffed67d050f9ad88552ecf4" exitCode=0 Sep 10 18:49:39.202525 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:49:39.202514 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" event={"ID":"b95e9aa7-c088-4327-b3c2-f659cb7f34ae","Type":"ContainerDied","Data":"a2e1d2c39baaae733bfe2dd6de7677d8a29de21f1ffed67d050f9ad88552ecf4"} Sep 10 18:49:39.202760 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:49:39.202570 2564 scope.go:117] "RemoveContainer" containerID="defc2a5ec8f69e23d03e79bfae067a54fe736384f4b3bcf0ad735f5d7f3488d2" Sep 10 18:49:39.202888 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:49:39.202870 2564 scope.go:117] "RemoveContainer" containerID="a2e1d2c39baaae733bfe2dd6de7677d8a29de21f1ffed67d050f9ad88552ecf4" Sep 10 18:49:39.203069 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:49:39.203053 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:49:41.284547 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:49:41.284511 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:49:41.284547 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:49:41.284554 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:49:41.284967 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:49:41.284565 2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:49:41.284967 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:49:41.284574 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" Sep 10 18:49:41.284967 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:49:41.284882 2564 scope.go:117] "RemoveContainer" containerID="a2e1d2c39baaae733bfe2dd6de7677d8a29de21f1ffed67d050f9ad88552ecf4" Sep 10 18:49:41.285118 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:49:41.285034 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:49:51.444093 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:49:51.444061 2564 scope.go:117] "RemoveContainer" containerID="a2e1d2c39baaae733bfe2dd6de7677d8a29de21f1ffed67d050f9ad88552ecf4" Sep 10 18:49:51.444598 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:49:51.444240 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:50:03.448154 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:03.448115 2564 scope.go:117] "RemoveContainer" containerID="a2e1d2c39baaae733bfe2dd6de7677d8a29de21f1ffed67d050f9ad88552ecf4" Sep 10 18:50:03.448692 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:50:03.448348 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:50:15.250129 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:15.250095 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bb8cfcdd6-28jxk_0b6bccde-cc93-4199-9007-68fba5ed46cd/manager/0.log" Sep 10 18:50:15.255306 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:15.255284 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bb8cfcdd6-28jxk_0b6bccde-cc93-4199-9007-68fba5ed46cd/copy-manifests/0.log" Sep 10 18:50:17.442529 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:17.442468 2564 scope.go:117] "RemoveContainer" containerID="a2e1d2c39baaae733bfe2dd6de7677d8a29de21f1ffed67d050f9ad88552ecf4" Sep 10 18:50:17.442999 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:50:17.442710 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:50:17.456550 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:17.456525 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-8jpqf_e8b49aaa-7628-48c6-823d-1122be9ca71e/manager/0.log" Sep 10 18:50:17.571322 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:17.571296 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-ckfpc_bf301dad-e02f-427c-bf5f-b927271f1b35/limitador/0.log" Sep 10 18:50:18.374171 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:18.374141 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64dfbcccbf-mmw7w_e43c3f93-a82d-4f05-a2e3-48ac25b6efae/kube-auth-proxy/0.log" Sep 10 18:50:19.571569 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:19.571535 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-fv4bp_8f9db1a3-5d09-4bc4-914f-8f5c1f3b62a7/main/0.log" Sep 10 18:50:19.931850 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:19.931771 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_b95e9aa7-c088-4327-b3c2-f659cb7f34ae/main/5.log" Sep 10 18:50:26.123423 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:26.123385 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ddcqf/must-gather-jsq8q"] Sep 10 18:50:26.126998 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:26.126977 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ddcqf/must-gather-jsq8q" Sep 10 18:50:26.129102 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:26.129074 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ddcqf\"/\"default-dockercfg-bf9tb\"" Sep 10 18:50:26.129288 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:26.129275 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ddcqf\"/\"openshift-service-ca.crt\"" Sep 10 18:50:26.129707 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:26.129691 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ddcqf\"/\"kube-root-ca.crt\"" Sep 10 18:50:26.134388 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:26.134367 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ddcqf/must-gather-jsq8q"] Sep 10 18:50:26.195281 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:26.195252 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/97b49c9d-0c2c-43f2-a4ca-ba3661e2bc4c-must-gather-output\") pod \"must-gather-jsq8q\" (UID: \"97b49c9d-0c2c-43f2-a4ca-ba3661e2bc4c\") " pod="openshift-must-gather-ddcqf/must-gather-jsq8q" Sep 10 18:50:26.195458 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:26.195363 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5dbm\" (UniqueName: \"kubernetes.io/projected/97b49c9d-0c2c-43f2-a4ca-ba3661e2bc4c-kube-api-access-x5dbm\") pod \"must-gather-jsq8q\" (UID: \"97b49c9d-0c2c-43f2-a4ca-ba3661e2bc4c\") " pod="openshift-must-gather-ddcqf/must-gather-jsq8q" Sep 10 18:50:26.296821 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:26.296786 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5dbm\" (UniqueName: \"kubernetes.io/projected/97b49c9d-0c2c-43f2-a4ca-ba3661e2bc4c-kube-api-access-x5dbm\") pod \"must-gather-jsq8q\" (UID: \"97b49c9d-0c2c-43f2-a4ca-ba3661e2bc4c\") " pod="openshift-must-gather-ddcqf/must-gather-jsq8q" Sep 10 18:50:26.296992 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:26.296844 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/97b49c9d-0c2c-43f2-a4ca-ba3661e2bc4c-must-gather-output\") pod \"must-gather-jsq8q\" (UID: \"97b49c9d-0c2c-43f2-a4ca-ba3661e2bc4c\") " pod="openshift-must-gather-ddcqf/must-gather-jsq8q" Sep 10 18:50:26.297138 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:26.297120 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/97b49c9d-0c2c-43f2-a4ca-ba3661e2bc4c-must-gather-output\") pod \"must-gather-jsq8q\" (UID: \"97b49c9d-0c2c-43f2-a4ca-ba3661e2bc4c\") " pod="openshift-must-gather-ddcqf/must-gather-jsq8q" Sep 10 18:50:26.321712 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:26.321687 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5dbm\" (UniqueName: \"kubernetes.io/projected/97b49c9d-0c2c-43f2-a4ca-ba3661e2bc4c-kube-api-access-x5dbm\") pod \"must-gather-jsq8q\" (UID: \"97b49c9d-0c2c-43f2-a4ca-ba3661e2bc4c\") " pod="openshift-must-gather-ddcqf/must-gather-jsq8q" Sep 10 18:50:26.436442 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:26.436349 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ddcqf/must-gather-jsq8q" Sep 10 18:50:26.704792 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:26.704767 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ddcqf/must-gather-jsq8q"] Sep 10 18:50:26.706623 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:50:26.706596 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97b49c9d_0c2c_43f2_a4ca_ba3661e2bc4c.slice/crio-d3bb86672bd69023847af85a9a40aa22ec1116d157323b34cfe8d59d367dc8c1 WatchSource:0}: Error finding container d3bb86672bd69023847af85a9a40aa22ec1116d157323b34cfe8d59d367dc8c1: Status 404 returned error can't find the container with id d3bb86672bd69023847af85a9a40aa22ec1116d157323b34cfe8d59d367dc8c1 Sep 10 18:50:27.448989 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:27.448954 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ddcqf/must-gather-jsq8q" event={"ID":"97b49c9d-0c2c-43f2-a4ca-ba3661e2bc4c","Type":"ContainerStarted","Data":"d3bb86672bd69023847af85a9a40aa22ec1116d157323b34cfe8d59d367dc8c1"} Sep 10 18:50:28.453561 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:28.453523 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ddcqf/must-gather-jsq8q" event={"ID":"97b49c9d-0c2c-43f2-a4ca-ba3661e2bc4c","Type":"ContainerStarted","Data":"654b0d48ff1e9eee206dce422c3d545aa29f7e4395d54a91e46c2889b1acd39d"} Sep 10 18:50:28.454106 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:28.454077 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ddcqf/must-gather-jsq8q" event={"ID":"97b49c9d-0c2c-43f2-a4ca-ba3661e2bc4c","Type":"ContainerStarted","Data":"df078ab43d9e9f2416b68adaa75a4f958c33e75ebf00dd3a31b7ca54db8322bf"} Sep 10 18:50:28.468778 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:28.468732 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ddcqf/must-gather-jsq8q" podStartSLOduration=1.598845325 podStartE2EDuration="2.468718618s" totalImagesPullingTime="869.873293ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:50:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:50:28.467854205 +0000 UTC m=+1157.611157468" watchObservedRunningTime="2026-09-10 18:50:28.468718618 +0000 UTC m=+1157.612021862" Sep 10 18:50:29.164550 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:29.164519 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mm2nc_25dec05c-c81f-4c62-a6e8-1d3506812ff5/global-pull-secret-syncer/0.log" Sep 10 18:50:29.252547 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:29.252516 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-q4chw_97385966-6ff5-44b8-9a49-cfb01d67fefe/konnectivity-agent/0.log" Sep 10 18:50:29.314479 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:29.314449 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-87.ec2.internal_7fc4c017da0da160b512c920380689c3/haproxy/0.log" Sep 10 18:50:31.457805 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:31.457772 2564 scope.go:117] "RemoveContainer" containerID="a2e1d2c39baaae733bfe2dd6de7677d8a29de21f1ffed67d050f9ad88552ecf4" Sep 10 18:50:31.458276 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:50:31.458052 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:50:34.527967 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:34.527910 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-8jpqf_e8b49aaa-7628-48c6-823d-1122be9ca71e/manager/0.log" Sep 10 18:50:34.551501 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:34.551458 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-ckfpc_bf301dad-e02f-427c-bf5f-b927271f1b35/limitador/0.log" Sep 10 18:50:36.073911 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.073875 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_afff2259-85d9-4d94-872e-da941e8fdc9a/alertmanager/0.log" Sep 10 18:50:36.094898 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.094858 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_afff2259-85d9-4d94-872e-da941e8fdc9a/config-reloader/0.log" Sep 10 18:50:36.117958 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.117932 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_afff2259-85d9-4d94-872e-da941e8fdc9a/kube-rbac-proxy-web/0.log" Sep 10 18:50:36.137774 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.137749 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_afff2259-85d9-4d94-872e-da941e8fdc9a/kube-rbac-proxy/0.log" Sep 10 18:50:36.162343 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.162280 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_afff2259-85d9-4d94-872e-da941e8fdc9a/kube-rbac-proxy-metric/0.log" Sep 10 18:50:36.187082 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.186997 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_afff2259-85d9-4d94-872e-da941e8fdc9a/prom-label-proxy/0.log" Sep 10 18:50:36.208881 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.208849 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_afff2259-85d9-4d94-872e-da941e8fdc9a/init-config-reloader/0.log" Sep 10 18:50:36.389037 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.388915 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-26vff_c3b212e2-aa57-4cae-a509-d4cd8a8e133a/monitoring-plugin/0.log" Sep 10 18:50:36.420534 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.420500 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dp8n2_32dba368-9cba-42ac-80bc-ee4ebe358f7a/node-exporter/0.log" Sep 10 18:50:36.447272 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.447247 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dp8n2_32dba368-9cba-42ac-80bc-ee4ebe358f7a/kube-rbac-proxy/0.log" Sep 10 18:50:36.473464 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.473428 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dp8n2_32dba368-9cba-42ac-80bc-ee4ebe358f7a/init-textfile/0.log" Sep 10 18:50:36.765064 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.765017 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fe3490c4-a7a9-4e07-9302-af7790f1f68b/prometheus/0.log" Sep 10 18:50:36.794076 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.794026 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fe3490c4-a7a9-4e07-9302-af7790f1f68b/config-reloader/0.log" Sep 10 18:50:36.818188 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.818158 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fe3490c4-a7a9-4e07-9302-af7790f1f68b/thanos-sidecar/0.log" Sep 10 18:50:36.838600 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.838563 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fe3490c4-a7a9-4e07-9302-af7790f1f68b/kube-rbac-proxy-web/0.log" Sep 10 18:50:36.859159 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.859130 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fe3490c4-a7a9-4e07-9302-af7790f1f68b/kube-rbac-proxy/0.log" Sep 10 18:50:36.885878 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.885844 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fe3490c4-a7a9-4e07-9302-af7790f1f68b/kube-rbac-proxy-thanos/0.log" Sep 10 18:50:36.903781 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.903756 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fe3490c4-a7a9-4e07-9302-af7790f1f68b/init-config-reloader/0.log" Sep 10 18:50:36.931505 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.931455 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-94vpm_4601784a-6817-45a3-b088-1c08b9410a01/prometheus-operator/0.log" Sep 10 18:50:36.954397 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.954364 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-94vpm_4601784a-6817-45a3-b088-1c08b9410a01/kube-rbac-proxy/0.log" Sep 10 18:50:36.980553 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:36.980462 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-gfdtb_0c39f34d-a437-4526-9abb-3cfe2d806bd4/prometheus-operator-admission-webhook/0.log" Sep 10 18:50:37.010592 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:37.010560 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f59bbb4cb-wvk9c_73e86922-fc64-445d-a85f-0f0c4b849ac2/telemeter-client/0.log" Sep 10 18:50:37.036806 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:37.036771 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f59bbb4cb-wvk9c_73e86922-fc64-445d-a85f-0f0c4b849ac2/reload/0.log" Sep 10 18:50:37.056800 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:37.056765 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f59bbb4cb-wvk9c_73e86922-fc64-445d-a85f-0f0c4b849ac2/kube-rbac-proxy/0.log" Sep 10 18:50:38.466427 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:38.466396 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-5tq6l_c6c62a12-04eb-40ce-b5cb-3a43fc562884/networking-console-plugin/0.log" Sep 10 18:50:39.070503 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.070453 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d28xz_3c15c29b-edff-4386-a50a-177b41560acd/console-operator/2.log" Sep 10 18:50:39.076338 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.076301 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d28xz_3c15c29b-edff-4386-a50a-177b41560acd/console-operator/3.log" Sep 10 18:50:39.134722 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.134678 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh"] Sep 10 18:50:39.141791 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.141755 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.146953 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.146923 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh"] Sep 10 18:50:39.240040 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.239984 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/98a3a762-f28c-42a9-8c2a-288d3d4254f9-podres\") pod \"perf-node-gather-daemonset-xndxh\" (UID: \"98a3a762-f28c-42a9-8c2a-288d3d4254f9\") " pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.240332 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.240049 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8gxr\" (UniqueName: \"kubernetes.io/projected/98a3a762-f28c-42a9-8c2a-288d3d4254f9-kube-api-access-m8gxr\") pod \"perf-node-gather-daemonset-xndxh\" (UID: \"98a3a762-f28c-42a9-8c2a-288d3d4254f9\") " pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.240332 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.240116 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98a3a762-f28c-42a9-8c2a-288d3d4254f9-sys\") pod \"perf-node-gather-daemonset-xndxh\" (UID: \"98a3a762-f28c-42a9-8c2a-288d3d4254f9\") " pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.240332 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.240146 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/98a3a762-f28c-42a9-8c2a-288d3d4254f9-proc\") pod \"perf-node-gather-daemonset-xndxh\" (UID: \"98a3a762-f28c-42a9-8c2a-288d3d4254f9\") " pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.240332 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.240202 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98a3a762-f28c-42a9-8c2a-288d3d4254f9-lib-modules\") pod \"perf-node-gather-daemonset-xndxh\" (UID: \"98a3a762-f28c-42a9-8c2a-288d3d4254f9\") " pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.340814 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.340715 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/98a3a762-f28c-42a9-8c2a-288d3d4254f9-podres\") pod \"perf-node-gather-daemonset-xndxh\" (UID: \"98a3a762-f28c-42a9-8c2a-288d3d4254f9\") " pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.340814 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.340764 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8gxr\" (UniqueName: \"kubernetes.io/projected/98a3a762-f28c-42a9-8c2a-288d3d4254f9-kube-api-access-m8gxr\") pod \"perf-node-gather-daemonset-xndxh\" (UID: \"98a3a762-f28c-42a9-8c2a-288d3d4254f9\") " pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.341044 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.340826 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98a3a762-f28c-42a9-8c2a-288d3d4254f9-sys\") pod \"perf-node-gather-daemonset-xndxh\" (UID: \"98a3a762-f28c-42a9-8c2a-288d3d4254f9\") " pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.341044 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.340858 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/98a3a762-f28c-42a9-8c2a-288d3d4254f9-proc\") pod \"perf-node-gather-daemonset-xndxh\" (UID: \"98a3a762-f28c-42a9-8c2a-288d3d4254f9\") " pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.341044 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.340912 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98a3a762-f28c-42a9-8c2a-288d3d4254f9-lib-modules\") pod \"perf-node-gather-daemonset-xndxh\" (UID: \"98a3a762-f28c-42a9-8c2a-288d3d4254f9\") " pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.341196 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.341119 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98a3a762-f28c-42a9-8c2a-288d3d4254f9-lib-modules\") pod \"perf-node-gather-daemonset-xndxh\" (UID: \"98a3a762-f28c-42a9-8c2a-288d3d4254f9\") " pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.341252 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.341203 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/98a3a762-f28c-42a9-8c2a-288d3d4254f9-podres\") pod \"perf-node-gather-daemonset-xndxh\" (UID: \"98a3a762-f28c-42a9-8c2a-288d3d4254f9\") " pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.341304 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.341253 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/98a3a762-f28c-42a9-8c2a-288d3d4254f9-proc\") pod \"perf-node-gather-daemonset-xndxh\" (UID: \"98a3a762-f28c-42a9-8c2a-288d3d4254f9\") " pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.341304 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.341286 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98a3a762-f28c-42a9-8c2a-288d3d4254f9-sys\") pod \"perf-node-gather-daemonset-xndxh\" (UID: \"98a3a762-f28c-42a9-8c2a-288d3d4254f9\") " pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.348722 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.348686 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8gxr\" (UniqueName: \"kubernetes.io/projected/98a3a762-f28c-42a9-8c2a-288d3d4254f9-kube-api-access-m8gxr\") pod \"perf-node-gather-daemonset-xndxh\" (UID: \"98a3a762-f28c-42a9-8c2a-288d3d4254f9\") " pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.458084 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.457823 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:39.686536 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.686440 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-xknxg_9f9b30af-9874-4e85-9843-b0997e177371/download-server/0.log" Sep 10 18:50:39.758311 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:39.758275 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh"] Sep 10 18:50:39.761844 ip-10-0-138-87 kubenswrapper[2564]: W0910 18:50:39.761811 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod98a3a762_f28c_42a9_8c2a_288d3d4254f9.slice/crio-2b6274ae2e2a677a5af8e457e05f65a89ef75d54b40fac2c173c37cc3e25f468 WatchSource:0}: Error finding container 2b6274ae2e2a677a5af8e457e05f65a89ef75d54b40fac2c173c37cc3e25f468: Status 404 returned error can't find the container with id 2b6274ae2e2a677a5af8e457e05f65a89ef75d54b40fac2c173c37cc3e25f468 Sep 10 18:50:40.373019 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:40.372986 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-ggp6z_5da5cd5f-ac01-4cf5-868b-ed1f600ef8d5/volume-data-source-validator/0.log" Sep 10 18:50:40.545147 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:40.545103 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" event={"ID":"98a3a762-f28c-42a9-8c2a-288d3d4254f9","Type":"ContainerStarted","Data":"d26cc99610894b87c247cd9a219a6d64180aa2b870259d934f4cbdc3dff4a3f2"} Sep 10 18:50:40.545329 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:40.545174 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" event={"ID":"98a3a762-f28c-42a9-8c2a-288d3d4254f9","Type":"ContainerStarted","Data":"2b6274ae2e2a677a5af8e457e05f65a89ef75d54b40fac2c173c37cc3e25f468"} Sep 10 18:50:40.545389 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:40.545357 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:40.560223 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:40.560164 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" podStartSLOduration=1.560144242 podStartE2EDuration="1.560144242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:50:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:50:40.558873883 +0000 UTC m=+1169.702177127" watchObservedRunningTime="2026-09-10 18:50:40.560144242 +0000 UTC m=+1169.703447487" Sep 10 18:50:41.428407 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:41.428375 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nggdz_857e07ef-d71a-41f6-b863-b90126b0febb/dns/0.log" Sep 10 18:50:41.446603 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:41.446578 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nggdz_857e07ef-d71a-41f6-b863-b90126b0febb/kube-rbac-proxy/0.log" Sep 10 18:50:41.469978 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:41.469947 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4lr7x_e4bb2ec5-efc3-4db5-a9e9-d0e7ba7e72a0/dns-node-resolver/0.log" Sep 10 18:50:42.105800 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:42.105774 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cd2ls_96332d76-e827-4612-a6b5-bd9d31fe347e/node-ca/0.log" Sep 10 18:50:43.140649 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:43.140612 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64dfbcccbf-mmw7w_e43c3f93-a82d-4f05-a2e3-48ac25b6efae/kube-auth-proxy/0.log" Sep 10 18:50:43.865158 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:43.865131 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zkgwb_cc9720ca-244c-4b07-88e5-e95fbeae172d/serve-healthcheck-canary/0.log" Sep 10 18:50:44.388087 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:44.388054 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jxt56_9eb589d3-13f2-4138-948a-373b4e6b8dff/kube-rbac-proxy/0.log" Sep 10 18:50:44.405748 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:44.405716 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jxt56_9eb589d3-13f2-4138-948a-373b4e6b8dff/exporter/0.log" Sep 10 18:50:44.423284 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:44.423256 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jxt56_9eb589d3-13f2-4138-948a-373b4e6b8dff/extractor/0.log" Sep 10 18:50:45.442468 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:45.442438 2564 scope.go:117] "RemoveContainer" containerID="a2e1d2c39baaae733bfe2dd6de7677d8a29de21f1ffed67d050f9ad88552ecf4" Sep 10 18:50:45.443031 ip-10-0-138-87 kubenswrapper[2564]: E0910 18:50:45.442718 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=premium-simulated-simulated-premium-kserve-5758cf6949-fvscb_llm(b95e9aa7-c088-4327-b3c2-f659cb7f34ae)\"" pod="llm/premium-simulated-simulated-premium-kserve-5758cf6949-fvscb" podUID="b95e9aa7-c088-4327-b3c2-f659cb7f34ae" Sep 10 18:50:46.566894 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:46.566860 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ddcqf/perf-node-gather-daemonset-xndxh" Sep 10 18:50:46.681043 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:46.681012 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bb8cfcdd6-28jxk_0b6bccde-cc93-4199-9007-68fba5ed46cd/manager/0.log" Sep 10 18:50:46.698308 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:46.698274 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bb8cfcdd6-28jxk_0b6bccde-cc93-4199-9007-68fba5ed46cd/copy-manifests/0.log" Sep 10 18:50:48.031258 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:48.031219 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-ddb978bf8-bzbnz_fad3eec9-27b2-496e-8be5-29c592111646/manager/0.log" Sep 10 18:50:48.081462 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:48.081328 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-cpvjh_4750f6be-a2e4-4a80-9347-d4afa70f2105/openshift-lws-operator/0.log" Sep 10 18:50:50.035655 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:50.035620 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64dfbcccbf-mmw7w_e43c3f93-a82d-4f05-a2e3-48ac25b6efae/kube-auth-proxy/0.log" Sep 10 18:50:51.411860 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:51.411831 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64dfbcccbf-mmw7w_e43c3f93-a82d-4f05-a2e3-48ac25b6efae/kube-auth-proxy/0.log" Sep 10 18:50:53.341233 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:53.341187 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-s6687_eba65242-6e91-4208-91db-6f590e66aebc/migrator/0.log" Sep 10 18:50:53.359024 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:53.358995 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-s6687_eba65242-6e91-4208-91db-6f590e66aebc/graceful-termination/0.log" Sep 10 18:50:53.711384 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:53.711295 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-sm6d9_418a0ba4-c771-43aa-bc0c-baf2b0b4f34f/kube-storage-version-migrator-operator/1.log" Sep 10 18:50:53.712147 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:53.712130 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-sm6d9_418a0ba4-c771-43aa-bc0c-baf2b0b4f34f/kube-storage-version-migrator-operator/0.log" Sep 10 18:50:54.632283 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:54.632248 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6jw7x_3e20510c-c392-4fc1-a566-e2b9179d3e08/kube-multus/0.log" Sep 10 18:50:54.815258 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:54.815225 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nrlp8_92d1a4d6-3b6a-4676-b798-8ca176789fb1/kube-multus-additional-cni-plugins/0.log" Sep 10 18:50:54.833840 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:54.833807 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nrlp8_92d1a4d6-3b6a-4676-b798-8ca176789fb1/egress-router-binary-copy/0.log" Sep 10 18:50:54.851851 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:54.851825 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nrlp8_92d1a4d6-3b6a-4676-b798-8ca176789fb1/cni-plugins/0.log" Sep 10 18:50:54.869019 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:54.868994 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nrlp8_92d1a4d6-3b6a-4676-b798-8ca176789fb1/bond-cni-plugin/0.log" Sep 10 18:50:54.886527 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:54.886426 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nrlp8_92d1a4d6-3b6a-4676-b798-8ca176789fb1/routeoverride-cni/0.log" Sep 10 18:50:54.903681 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:54.903657 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nrlp8_92d1a4d6-3b6a-4676-b798-8ca176789fb1/whereabouts-cni-bincopy/0.log" Sep 10 18:50:54.921771 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:54.921748 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nrlp8_92d1a4d6-3b6a-4676-b798-8ca176789fb1/whereabouts-cni/0.log" Sep 10 18:50:55.114397 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:55.114335 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6dxjk_99da651c-7b27-49ea-b49a-7f72caf21100/network-metrics-daemon/0.log" Sep 10 18:50:55.131053 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:55.131026 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6dxjk_99da651c-7b27-49ea-b49a-7f72caf21100/kube-rbac-proxy/0.log" Sep 10 18:50:56.328191 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:56.328099 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/ovn-controller/0.log" Sep 10 18:50:56.342719 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:56.342669 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/ovn-acl-logging/0.log" Sep 10 18:50:56.342923 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:56.342905 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/ovn-acl-logging/1.log" Sep 10 18:50:56.361245 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:56.361221 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/kube-rbac-proxy-node/0.log" Sep 10 18:50:56.380048 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:56.380005 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/kube-rbac-proxy-ovn-metrics/0.log" Sep 10 18:50:56.395217 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:56.395195 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/northd/0.log" Sep 10 18:50:56.415875 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:56.415850 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/nbdb/0.log" Sep 10 18:50:56.447263 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:56.447233 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/sbdb/0.log" Sep 10 18:50:56.505294 ip-10-0-138-87 kubenswrapper[2564]: I0910 18:50:56.505256 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pb2pr_019680e6-9104-47fb-b272-9ced99c8bfc8/ovnkube-controller/0.log"